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CHAPTER  ONE 

Introduction 


Background 

Over  time,  the  Department  of  Defense  (DoD)  has  institutionalized  a  yearly  munitions  planning 
process  that  plans  for  specific  wars  and  results  in  the  accumulation  of  large  munitions  stocks.  It  is 
called  the  DoD  Munitions  Requirements  (MR)  process.  The  institution’s  creation  made  sense  in  the 
context  of  the  cold  war  —  where  the  threat  was  of  an  immediate  and  tremendous  loss  (50,000  tanks 
crossing  into  Europe)  that  could  be  countered  by  geographically  specific  worst  case  planning. 

The  MR  process  does  not  make  as  much  sense  today.  The  DoD  faces  increasing  uncertainty  about 
both  the  threats  it  will  face  and  the  amount  of  money  it  will  receive  to  prepare  for  them.  The 
requirements  process  predicts  demand  for  munitions  based  on  specific  scenarios,  and  does  not 
include  cost  as  an  integral  factor.  The  resulting  demand,  called  the  ‘requirement,’  is  used  to  inform 
and  justify  munitions  acquisitions,  and  shapes  the  structure  of  the  industrial  base.  Such  a  process 
that  takes  mission,  demand,  and  funding  for  given  is  perhaps  no  longer  a  good  match  to  today’s 
reality.  Furthermore,  today’s  rapid  technological  advances  and  diverse  engagements  have  quickly 
changed  tactics  and  weapon  preferences  —  and  can  be  expected  to  do  so  in  the  future. 

The  DoD  recognizes  the  need  to  fundamentally  change  the  way  it  plans  for  wars  and  makes 
purchasing  decisions,  and  has  recently  begun  to  promote  a  capabilities-based  planning  approach  over 
traditional  requirements  based  approaches.  As  defined  by  Paul  Davis  (emphasis  added): 

Capabilities-based  planning  is  planning,  under  uncertainty,  to  provide  capabilities  suitable  for  a  wide  range 
of  modern  day  challenges  and  circumstances,  while  working  within  an  economic  framework. ' 

However,  as  of  the  writing  of  this  paper,  such  an  approach  has  not  been  applied  to  complex  problem 
of  munitions  planning. 


Motivation 

As  a  result  of  recent  criticisms  by  the  Government  Accountability  Office  (GAO)  and  DoD  Inspector 
General  and  a  department  wide  push  for  the  use  of  new  planning  methods,  the  DoD  is  re-examining 
the  Munitions  Requirements  process.  Concurrently,  the  Office  of  the  Secretary  of  Defense  (OSD)  is 
looking  for  ways  to  make  its  precision  guided  munitions  supply  more  “flexible  and  responsive”  to  the 
needs  of  policy  makers  and  war-fighters.  Precision  Guided  Munitions  (PGM)  stocks  and 
expenditures  are  more  sensitive  to  the  planning  process  than  traditional  munitions  because  they  are 
more  expensive  and  complex,  tend  to  be  built  by  private  subcontractors,  are  in  high  demand  and 
have  a  short  shelf  life  relative  to  traditional  unguided  munitions.  On  top  of  this,  PGMs  are  rapidly 


'  Davis,  Paul.  “Analytic  Architecture  for  Capabilities-Based  Planning,  Mission-System  Analysis,  and  Transformation”  RAND,  Santa 
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becoming  a  staple  rather  than  a  niche  weapon.  In  Desert  Storm,  PGMs  accounted  for  8%  of  the 
munitions  fired.  They  accounted  for  over  60%,  and  possibly  as  much  as  90%,  of  the  weapons  fired 
during  Operation  Iraqi  Freedom. 

Institutional  Movement  Toward  Capabilities-based  Planning 

The  DoD  has,  in  the  past,  tried  to  improve  its  requirements  system  —  the  instruction  governing  the 
process  has  changed  three  times  during  the  last  seven  years.  Acquisition  reform  has  long  been  a  major 
issue  for  OSD  and  Congressional  leadership.  But  over  the  last  decade  there  have  been  many  obstacles 
to  change  present  within  the  DoD.  Often  cited  obstacles  include  organizational  inertia,  ‘stove-piped’ 
management,  service  oriented  processes,  and  the  presence  of  a  decentralized  power  structure. 

Military  transformation  has  been  slow,  but  new  ways  of  thinking  are  permeating  the  DoD.  The  old 
paradigm  has  been  to  plan  based  on  what  Paul  Davis  calls  ‘point-scenarios.’  According  to  Davis, 
point  scenario  planning  fixates  on  “particular  enemies,  particular  wars,  and  particular  assumptions 
about  those  wars  —  a  fixation  that  comes  at  the  expense  of  more  flexible  and  adaptive  planning.”2 
High-level  policymakers  have  become  increasingly  aware  of  the  importance  of  adaptability.  The 
concept  of  capabilities-based  planning  is  now  moving  rapidly  from  idea  to  practice.  The  2001 
Quadrennial  Defense  Review  announced  a  new  defense  strategy  based  upon  a  “capabilities-based 
approach  to  defense.”3  The  recently  completed  Joint  Defense  Capabilities  Study  recommends  a  joint 
process,  centrally  managed  by  Department  of  Defense  and  overseen  by  the  Secretary  of  Defense.4 
There  have  been  obstacles  in  the  past,  but  todayAfs  leaders  recognize  the  problems  and  openly 
voice  a  strong  desire  to  reform  the  planning  and  acquisition  process. 

Approach 

This  dissertation  provides  an  example  of  how  the  ideas  of  capabilities-based  planning  can  be  applied 
to  the  problem  of  PGM  planning  in  the  form  of  an  illustrative,  or  demonstrative,  analysis.  Two 
interrelated  methodologies  are  used  in  this  illustrative  analysis:  exploratory  modeling  and  robust 
planning.  Recall  that  capabilities-based  planning  seeks  to  find  capabilities  that  are  suitable  for  a  wide 
range  of  uncertain  circumstances,  while  simultaneously  working  within  an  economic  framework. 

Phis  research  shows  that  by  applying  the  methodology  of  exploratory  analysis  to  a  single  war  scenario, 
we  may  come  to  a  more  complete  understanding  of  how  changes  in  the  constraints  of  cost  and 
production  capacity  and  the  uncertainty  of  preparation  time  can  drastically  affect  the  performance  of 
a  munitions  portfolio.  The  presence  of  multiple  war  scenarios  complicates  this  sort  of  analysis,  and 
makes  it  impossible  to  find  a  single  optimal  munitions  portfolio.  We  show  that  the  use  of  robust 
decision-making  methodologies  makes  it  possible  to  analyze  the  effects  of  these 


2  Davis,  Paul.  “Analytic  Architecture  of  Capabilities-Based  Planning,  Mission-System  Analysis,  and  Transformation.”  RAND:  Santa 
Monica  CA,  2002.  p.  8 

^Office  of  the  Secretary  of  Defense.  “2001  Quadrennial  Defense  Review  Report”  Washington,  DC,  2001.  p.  13 
^  Joint  Defense  Capabilities  Study.  December  2003.  p.3 
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constraints/uncertainties  across  a  wide  variety  of  scenarios  thus  allowing  us  to  create  improved  and 
flexible  munitions  portfolios. 

By  accepting  uncertainties  and  economic  constraints  and  dealing  with  them  in  a  sophisticated 
manner,  we  come  to  a  more  complete  understanding  of  the  strategy  chosen,  possible  alternatives,  and 
the  effects  of  changes  in  the  constraints.  Policymakers  will  be  able  to  see  and  understand  tradeoffs 
across  a  wide  variety  of  budget,  production,  preparation  and  threat  scenarios.  The  ability  to 
incorporate  this  kind  of  information  will  improve  the  decision  process  and  make  it  possible  to 
adequately  prepare  to  meet  a  wide  range  of  uncertain  threats  with  limited  resources. 

The  analysis  provided  here  is  meant  for  illustrative  purposes  only,  and  will  be  referred  to  as  the 
illustrative  or  demonstrative  analysis  throughout  the  body  of  this  paper.  Much  of  the  information 
used  to  determine  an  appropriate  weapons  mix  is  classified.  Weapon  effectiveness  data,  target  data 
and,  perhaps,  even  some  of  the  weapons  themselves  are  classified.  Only  unclassified  data  and  sources 
were  reviewed  in  the  preparation  of  this  report.  The  aim  is  not  to  provide  a  specific  answer,  but 
rather  to  show  how  a  capabilities-based  approach  may  be  applied  to  the  problem  and  demonstrate  the 
advantages  of  doing  so. 


Structure 

The  first  chapter  describes  the  planning  process  that  is  now  used  to  buy  precision-guided  weapons. 
The  Munitions  Requirements  process  is  defined.  This  chapter  includes  a  detailed  discussion  of 
documented  criticisms  of  the  process,  with  special  attention  paid  to  those  presented  by  the  GAO  and 
DoD  Inspector  General.  Such  criticisms  are  considered  overly  harsh  in  military  circles,  and  counter¬ 
arguments  are  presented  where  warranted. 

The  second  chapter  introduces  the  problem  as  one  of  resource  allocation.  It  argues  that  there  are 
several  things  the  DoD  can  do  differently  in  order  to  solve  the  munitions  planning  problem  if  it 
wishes  to  pursue  a  capabilities-based  approach.  Specifically,  the  DoD  can  deal  with  uncertainty 
explicitly  and  consider  the  economic  constraints  of  cost  and  production  capacity  concurrently  with 
procurement  decisions. 

The  third  chapter  expands  on  the  complex  nature  of  the  problem  through  a  brief  discussion  of  the 
two  most  relevant  academic  literatures:  resource  allocation  and  inventory  control.  The  chapter  argues 
that  pure  resource  management  methodologies  are  impractical  because  of  uncertainties  in  goals  and 
performance  measures  across  a  wide  variety  of  war  scenarios.  It  argues  that  pure  inventory  control 
methods  are  inadequate  because  of  the  erratic  nature  of  PGM  demand.  Again,  this  is  the  heart  of 
capabilities-based  planning  —  finding  solutions  that  work  across  a  wide  variety  of  scenarios  and 
uncertainties  while  operating  within  an  economic  framework. 

The  fourth  chapter  presents  the  methodologies  of  exploratory  analysis  and  robust/ adaptive  planning 
and  shows  how  they  are  particularly  suited  to  this  problem. 
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The  fifth  chapter  describes  the  form  of  the  illustrative  analysis  that  will  be  used  and  explains  how  it 
can  help  planners  better  understand  the  important  tradeoffs.  The  model  that  will  be  used  in  the 
illustrative  analysis  is  specified  here. 

The  sixth  chapter  uses  exploratory  analysis  to  show  that  the  constraints  of  cost,  production  and 
preparation  time  dramatically  affect  the  optimal  portfolio  of  weapons  for  a  single  scenario.  Then  it 
demonstrates  that  the  use  of  robust  decision-making  techniques  makes  it  possible  to  analyze  the 
effects  of  these  constraints  across  multiple  scenarios  to  create  improved  and  flexible  portfolios. 

The  seventh  chapter  presents  conclusions  to  be  drawn  and  offers  potential  extensions. 


CHAPTER  TWO 

The  Munitions  Requirements  Process 


Munitions  Requirements  Defined 

Conventional  munitions  requirements  are  “point  estimates  of  the  quantities  of  munitions  the  services 
would  buy  to  arm  their  planned  forces  if  sufficient  funds  were  available.”5  Although  the  estimates  are 
not  constrained  monetarily  in  a  single  process,  they  are  used  as  primary  data  to  inform  and  justify 
large-scale  munitions  procurement  decisions.  The  requirements  do  not  result  in  automatic  inventory 
changes,  but  are  used  by  munitions  planners  and  budgeters  in  entirely  separate  processes.  The 
military  services  are  responsible  for  having  enough  munitions  to  meet  the  current  requirement.6  The 
requirements  are  calculated  on  a  yearly  basis. 

The  calculation  of  the  requirement  clearly  uses  depends  on  classified  information.  Munitions 
effectiveness  data  are  classified.  Specific  targets  identified  in  the  process  are  classified.  Some  of  the 
munitions  used  may  be  classified.  However,  the  process  itself  is  not  classified  and  will  be  discussed  in 
great  detail  in  the  following  sections. 

The  total  munitions  requirement  is  comprised  of  a  war  reserve  requirement  and  a  testing,  training, 
and  current  operational  requirement.  The  war  reserve  portion  accounts  for  the  vast  majority  of  the 
total  requirement,  and  is  the  primary  subject  of  this  research.  War  reserve  munitions  requirements 
are  the  sum  of  three  types  of  requirements:  combat  requirements,  current  operations/forward 
presence  requirements,  and  strategic  readiness  requirements.  The  combat  requirement  is  the  quantity 
of  munitions  needed  to  equip  a  specific  force  structure  to  achieve  the  planned  wartime  objectives  of 
its  Combatant  Commander  (COCOM).  The  current  operations/forward  presence  requirement  is 
the  quantity  needed  to  equip  forces  that  are  not  committed  to  specific  war  plans  or  scenarios.  The 
strategic  readiness  requirement  is  the  quantity  needed  to  maintain  treaty  agreements  and  support 
forces  that  are  not  committed  to  operations  in  major  theaters  of  war.7 

The  documents  that  lay  out  current  munitions  requirements  policy  are  known  as  the  DoD  3000 
series.  The  process  created  by  these  documents  breaks  down  the  complex  problem  of  munitions 
planning  into  smaller  parts.  A  different  agency  is  responsible  for  each  piece  of  the  problem, 
beginning  with  the  Office  of  the  Undersecretary  of  Defense  for  Policy  and  Planning  and  ending  with 
each  of  the  services.  In  a  different  process  conducted  after  the  requirement  is  calculated,  each  service 
budgets  for  its  own  munitions  requirement  separately,  and  that  budget  is  fit  into  the  overall  DoD 
budget  by  the  Joint  Chiefs  in  a  complex  and  iterative  process  that  ends  when  final  approval  comes 
from  the  White  House. 


5  Kassing,  David  et  ai  “Estimating  Conventional  Munitions  Requirements”  RAND:  1991.  p.  1 

6  GAO/NSAID-96-129  p.  14 

7  OIG  2000  p.  2 
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Current  Stockpile  and  Annual  Expenditures 

This  ammunition,  if  loaded  onto  railroad  cars,  would  stretch  over  800  miles  -  the  distance  from 

Washington,  D.C.  to  Orlando,  Florida.  GAO 

In  1996,  the  General  Accounting  Office  estimated  that  the  services  carried  a  stockpile  of  over  5 
million  tons  of  conventional  ammunition  that  was  worth  approximately  $80  billion.8  The  Army 
alone  uses  over  35  million  square  feet  to  store  approximately  3  million  tons  of  stock.9  In  September 
of  2000,  the  Department  of  Defense  reported  an  inventory  of  munitions,  missile  systems  and  related 
equipment  valued  at  $76.8  billion.10  The  Department  spends  about  $5  billion  on  conventional 
munitions  every  year. 1 1  The  trend  towards  an  increased  reliance  on  precision  weapons  is  reflected  in 
DoD  munitions  budgets.  Annual  expenditures  for  precision-guided  weapons  were  $775  million  in 
1998,  with  a  planned  increase  to  $2  billion  per  year  by  2003. 12  Although  this  is  a  relatively  small 
portion  of  the  defense  budget,  it  can  be  argued  that  the  rest  of  the  budget  is  devoted  to  activities  that 
support  the  delivery  of  these  weapons  to  their  targets. 


The  Munitions  Requirement  Generation  Process  Defined 

The  formal  yearly  process  used  to  determine  the  munitions  requirement  is  called  the  DoD  Munitions 
Requirement  Process.  Over  the  last  seven  years,  there  have  been  three  different  versions  of  the  DoD 
Instruction  defining  the  process.13  The  name  of  the  process  has  changed,  but  changes  in  content 
appear  to  be  minimal.  The  document  provides  a  fairly  detailed  description  of  how  the  combat 
requirement  is  to  generated: 

Step  One:  Create  Scenarios  and  Find  Targets 

The  process  begins  when  the  Undersecretary  of  Defense  for  Policy  (USDP)  develops  the  Defense 
Planning  Guidance  document.  This  document  includes  specific  scenarios,  known  as  “Illustrative 
Planning  Scenarios,”  to  be  used  as  the  primary  input  of  the  Munitions  Requirement  process.  The 
USDP  is  tasked  to  “specify  critical  guidelines,  such  as  relevant  theaters  for  consideration  and  conflict 
duration.”14  The  Defense  Intelligence  Agency  (DIA)  creates  Threat  Reports  based  upon  the 
scenarios  provided  by  the  USDP.  The  threat  reports  are  target  lists.  These  lists  are  the  “authoritative 
threat  estimate  for  developing  munitions  requirements.”15 


8  GAO/NSIAD-96-129  “Significant  Problems  Left  Unattended  Will  Get  Worse”  June  1996.  p.  14 

9  Ibid.  p.  15 

10  Department  of  Defense.  “Supply  System  Inventory  Report”  September  30,  2000.  p.  15 

1 1  Office  of  the  Inspector  General  DoD.  “Summary  of  the  DoD  Process  for  Developing  Quantitative  Munitions  Requirements” 
February  24,  2000.  p.  3 

12  GAO/NSAID-99-32  “Guided  Weapons  Plans  need  to  be  Reassessed”  December  1998.  p.  4-5 

13  DoD  Instruction  3000.4.  The  different  versions  are  dated  June  16,  1997;  August  10,  2001;  and  October  23,  2003. 

14  DOD  Instruction  3000.4  October  23,  2003  p.4 

15  Ibid  p.  8 
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Step  Two:  Divide  Up  the  Targets 

The  Combatant  Commanders  responsible  for  regions  covered  in  the  Illustrative  Planning  Scenarios 
divide  up  the  targets  provided  in  the  Threat  Report  among  the  services  (Army,  Navy,  Air  Force, 
Marines).  The  targets  are  to  be  distributed  in  accordance  with  the  Commanders’  own  operational 
plans.  This  division  of  labor  between  the  Services  becomes  actionable  in  a  document  called  the 
Phased  Threat  Distribution  (PTD). 

Step  Three:  Generate  the  Requirement 

Each  service  is  responsible  for  calculating  the  requirement  for  its  portion  of  the  Phased  Threat 
Distribution.  The  calculations  are  done  by  a  wide  variety  of  mathematical  models.  Each  service  uses 
a  different  model,  and  in  many  cases,  uses  more  than  one.  In  most  cases,  the  models  take  about  a 
year  to  run.  The  aggregate  result  of  these  calculations  is  the  combat  munitions  requirement.  The 
most  recent  version  of  Instruction  3000.4  mandates  that  the  calculations  be  done  twice  —  once 
constrained  by  inventory,  and  once  without  any  constraint  other  than  “reasonable  production 
capacity,”  “  weapon  system  capabilities,”  and  “externally  defined  caps  on  procurement.”16  The 
current  operations/ forward  presence  and  strategic  readiness  requirements  are  also  calculated  by  each 
service  separately.  They  are  added  to  the  combat  requirement  to  generate  the  total  requirement. 


Cited  Problems  with  MR  Process 

Between  1994  and  1999,  over  20  reports  issued  by  the  GAO  and  DoD  Inspector  General’s  office 
identified  systemic  problems  within  the  munitions  requirements  process.  Easily  accessible  studies 
critical  of  the  process  date  back  to  1989,  and  the  GAO  has  issued  reports  as  recently  as  October 
2002.  The  more  recent  reports  not  only  identify  possible  problems,  but  also  claim  that  they  are  not 
being  fixed.  Additionally,  the  DoD  has  had  to  deal  with  both  the  problem  of  destroying  outdated 
and  unusable  munitions  and  the  problem  of  replenishing  newer  munitions  that  have  been  expended 
at  a  rate  greater  than  expected.  According  to  these  critiques,  recent  acquisition  plans  have  been 
justified  using  requirements  that  are  vastly  overstated  as  a  result  of  poor  coordination  between  the 
services  and  faulty  processes  within  the  system.  At  the  same  time,  we  have  faced  PGM  shortages 
during  each  major  conflict  since  the  Gulf  War. 

All  criticisms  may  be  summed  up  in  a  single  sentence:  “the  requirements  don’t  match  the  reality.” 
The  overarching  story  pieced  together  by  the  documents  is  as  follows.  The  segmented  requirements 
process  compartmentalizes  and  effectively  hides  some  of  the  uncertainty  faced  by  planners  in  each 
subsequent  step.  In  order  to  make  up  for  this,  planners  add  risk  premiums  to  their  estimates,  which 
in  the  end  creates  a  point  estimate  that  may  be  difficult  to  justify  to  accounting  agencies  and/or 
change  to  fit  available  resources  during  the  later  budgeting  process.  In  addition,  critics  attack  the 
sheer  numbers  of  methodologies  used,  length  of  the  process,  the  assumptions  used  and  the  sensitivity 
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of  models  to  these  assumptions.  The  following  sections  will  describe  how  critics  believe  the  MR 
process  prevents  planners  from  coming  to  a  coherent  understanding  of  the  true  risk  associated  with  a 
given  munitions  portfolio  or  the  tradeoffs  that  are  possible  in  the  production,  purchase,  use  and 
storage  of  munitions. 

Assumptions  at  Various  Steps  Can  Hide  the  Uncertainties 

Planners  responsible  for  each  step  of  the  requirements  process  must  make  certain  assumptions  in 
order  to  provide  the  documents  and  data  needed  in  subsequent  steps.  The  USDP  and  DIA  must 
decide  the  size,  type  and  duration  of  the  wars  for  which  we  should  prepare  so  that  the  combatant 
commanders  can  determine  which  strategies  they  will  employ,  which  result  in  the  force  mixes  to  be 
used.  The  services  must  build  weapon  choice  and  effectiveness  (among  other  things)  data  into  their 
simulations  and  models  that  represents  current  tactics  and  operational  concepts. 

We’d  like  to  know  for  certain  who  we  will  fight,  where  we  will  fight,  how  we  will  fight,  what  forces 
we  will  fight  with,  and  the  conditions  under  which  the  fighting  will  be  done.  The  DoD  recognizes 
that  in  today’s  world  we  have  much  less  certainty  about  such  things  than  we  did  20  or  30  years  ago  — 
hence  the  capabilities-based  planning  initiatives  currently  under  way.  But  such  an  approach  has  yet 
to  be  implemented  in  munitions  planning.  A  1991  RAND  review  of  the  MR  process  states  that 
“munitions  requirements  processes  treat  as  known  much  that  is,  in  fact,  unknowable”  and  concludes 
that  “effects  of  uncertainties  in  warfare  are  by  and  large  ignored  in  the  calculation  of  munitions 
requirements.”17  Rather  than  treat  uncertainty  as  a  core  element  of  munitions  planning,  each  step  of 
the  MR  process  tends  to  compartmentalize  the  uncertainty  through  a  simplifying  assumption  in 
order  to  solve  its  piece  of  the  problem.  This  limits  the  uncertainty  faced  in  subsequent  steps.  In 
many  cases,  these  assumptions  seem  perfectly  reasonable,  but  in  total  (as  the  system  works  now)  they 
preclude  a  complete  understanding  of  the  problem  at  hand  —  preparing  for  a  wide  variety  of  uncertain 
threats  within  the  context  of  an  economic  framework. 

The  resulting  requirement  is  a  best  estimate  made  under  certain  assumptions  that  limit  an  inherently 
large  degree  of  uncertainty.  The  requirement  is  sensitive  to  changes  in  those  assumptions.  The 
reality  of  the  type  and  duration  of  a  war  may  or  may  not  match  the  scenarios.  Consider  the  recent 
number  of  small-scale  contingencies  that  have  been  dealt  with  through  the  use  of  force  -  resulting  in 
the  use  of  munitions  that  were  bought  to  satisfy  a  different  and  particular  requirement.  Combatant 
commanders  may  or  may  not  change  their  strategies.  Field  commanders  may  or  may  not  make  the 
same  decisions  as  the  services’  models.  In  order  to  make  up  for  this,  responsible  planners  at  each 
level  do  what  they  can  to  guard  against  downside  risk  by  adding  a  risk  premium  to  their  estimates.  In 
the  aggregate,  this  results  in  what  critics  call  overly  inflated  requirements  and  potentially  severe 
overbuying  and  stocking  of  particular  munitions.  Although  carrying  a  large  amount  of  safety  stock 
may  be  a  very  good  thing,  it  appears  to  be  a  byproduct  of  the  planning  system  and  not  a  conclusion 
arrived  at  through  careful  analysis. 


17 


Kassing  et  al  p.2 


The  Munitions  Requirements  Process 


Requirements  are  Systematically  Distorted  to  Make  Up  for  Hidden  Uncertainty 

Critics  claim  that  those  tasked  with  running  the  process  inflate  the  requirement  at  every  level  and  in 
almost  every  conceivable  way.  The  result:  overstated  requirements  that  make  it  very  difficult  for  any 
single  person,  office,  or  group  understand  the  tradeoffs  possible  in  planning  or  the  character  of  risk 
underlying  the  required  munitions  portfolio.  Even  if  one  could,  according  to  the  GAO,  “the  DoD 
has  no  central  oversight  body  or  mechanism  to  examine  guided  weapons  programs  and  to  assess  how 
many  weapons  are  needed  to  meet  national  objectives  or  how  many  weapons  the  DoD  can  afford.”18 

Recall  that  the  process  begins  when  the  USDP  chooses  specific,  worst-case  scenarios,  creating  the 
Illustrative  Planning  Scenarios.  The  GAO  claims  that  the  USDP  continues  to  choose  the  worst  case 
despite  the  “DIA’s  projections  on  recent  international  trends  [(a  reduced  threat)],  the  sizable 
inventory  of  capable  weapons,  and  the  current  budgetary  situation.”19  The  USDP  gives  no 
information  that  would  indicate  the  likelihood  of  these  scenarios’  actual  occurrence.20  However,  the 
argument  for  planning  for  the  worst  case  is  compelling  -  if  we  have  enough  munitions  available  to 
face  even  the  worst  possible  adversary,  we  should  be  prepared  if  a  lesser  threat  arises.  This  does  not 
necessarily  mean  that  we  will  be  adequately  prepared  to  face  a  different  kind  of  threat. 

The  DIA  then  generates  a  comprehensive  target  list  for  the  regions  associated  with  the  (already  worst 
case)  scenarios  (the  Threat  Report).  The  COCOMs  and  services  use  almost  all  of  the  targets  as 
inputs  to  their  tasks,  even  though  it  is  widely  acknowledged  to  be  unlikely  that  more  than  a  small 
portion  of  them  will  actually  be  attacked.21  When  the  GAO  questioned  DoD  officials,  they 
acknowledged  that  including  nearly  all  the  targets  may  inflate  requirements,  but  said  that  they  did 
“not  want  to  risk  having  insufficient  weapons,  should  some  unforeseen  conflict  require  them.”22 
Again,  the  reasoning  behind  such  a  decision  is  fundamentally  sound  —  the  COCOMs  want  to  be 
absolutely  certain  that  they  are  able  to  fulfill  their  obligations.  If  they  plan  on  being  able  to  hit  all 
targets  in  a  region,  they  may  be  certain  that  the  capability  will  be  there  to  hit  any  combination  of 
targets. 

In  the  next  step,  the  COCOMs  create  the  Phased  Threat  Distribution  by  dividing  the  targets 
specified  in  the  Threat  Report  amongst  the  services.  The  COCOMs  routinely  divide  the  targets  in 
such  a  way  that  the  aggregate  proportion  of  assigned  targets  exceeds  100%.  The  GAO  cites  an 
example  of  one  COCOM  who,  with  the  objective  of  destroying  80%  of  a  target  type,  apportioned 
140%  of  the  targets  amongst  the  services.23  The  over  assignment  of  targets  is  justified  in  the  name 
of  flexibility.  COCOMs  want  to  be  sure  that  all  the  targets  can  be  hit,  even  if  plans  change  or  assets 


18  GAO/N SAID-99-32  p.  7 

19  GAO/N  SAID-99-32  p.  34 

20  GAO  -01-18  p.  9 

21  GAO/N  SAID-99-32  p.  34 

22  GAO/N  SAID-99-32  p.  35 

23  NSAID/GAO-96-72  p.  23 
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assigned  to  one  service  or  another  become  unavailable.  However,  the  GAO  sees  it  differently  and 
goes  so  far  as  to  say: 

Defense  Intelligence  Agency  officials  stated  that  in  the  past,  the  services  could,  based  on  input  from  their 
own  intelligence  sources,  or  input  from  the  war  fighting  Commanders  in  Chief  [(COCOMs)] ,  develop  an 
independent  threat  analysis  that  could  result  in  the  services  planning  to  destroy  the  same  targets,  and 
consequently,  overstating  munitions  requirements.24 

The  phased  threat  distribution  is  then  used  by  the  individual  services  as  inputs  for  their  own  detailed 
combat  models.  In  order  to  account  for  uncertainty,  the  services  have  been  known  to  both  add 
targets  before  running  the  models  -  and  add  munitions  afterwards.  In  its  criticism  of  the  DoD  Anti- 
Armor  Munitions  Plan,  the  GAO  identifies  four  specific  cases  of  requirements  inflation  at  the  service 
level:  The  Air  Force  increased  its  target  allocations  by  13-21  percent,  depending  on  the  target  type. 
The  Marine  Corps  and  Army  did  not  always  accept  their  methodologies’  results  and  manually 
calculated  higher  requirements.  The  services’  methodologies  proscribed  the  use  of  expensive,  high 
tech  anti-armor  weapons  against  unarmored  targets.  The  services  added  weapons  to  account  for 
uncertainties  —  the  most  shocking  example  being  that  only  3  percent  of  the  Army’s  direct  fire 
weapons  would  be  used  against  their  assigned  target.  The  remaining  97  percent  are  for  added 
flexibility  and  risk  mitigation.25  Furthermore,  invisible  margins  of  safety  may  be  found  inside  the 
models.  Many  assumptions  used  in  the  services’  models  tend  to  increase  the  requirement  in  ways  that 
are  nearly  invisible.  If  munitions’  kill  probabilities  are  too  low,  or  enemy  repair  rates  too  high,  the 
resulting  requirement  will  be  artificially  high.  Such  assumptions  take  the  form  of  parameters  buried 
in  the  services’  models. 

Other  Problems 

A  Myriad  of  Modeling  Methodologies 

The  models  used  by  the  services  have  a  myriad  of  problems  identified  over  the  last  10  years.  There  is 
no  consistent  methodology  used  to  determine  the  combat  requirement.  Each  service  uses  an  entirely 
different  analytic  method  and  most  use  more  than  one.  A  RAND  study  identified  seven  different 
methodologies  used  between  the  four  services.26  Kassing  et  al  go  to  great  lengths  to  describe  the 
major  differences.  The  Army  uses  very  detailed  combat  models  to  calculate  its  requirement,  while  the 
other  services’  models  are  less  specific.  Some  methodologies  set  requirements  based  on  the  number  of 
available  targets,  without  regard  for  the  services’  ability  to  expend  munitions.  Others  set 
requirements  based  on  the  ability  to  expend  munitions,  without  regard  for  the  number  of  targets  that 
may  be  available.  Each  service  has  a  different  way  of  calculating  the  confidence  level  of  its  estimate. 
Each  service  treats  re-supply  of  ammunition  in  a  different  way  (if  at  all).  Each  service  has  a  different 
way  of  including  cost  effectiveness  calculations  in  its  estimate  (rudimentary  or  not  at  all).  Some 
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services  assign  values  to  various  targets,  while  other  services  simply  count  the  number  of  targets.  A 
recent  report  by  the  DoD  Inspector  General  cites  further  inconsistencies  between  the  services’ 
methodologies.27  For  example,  two  services  included  combat  losses  in  their  requirements,  and  two 
did  not.  Only  one  service  considered  re-supply  in  its  calculations.  Each  service  had  a  different 
interpretation  of  the  current  operations/forward  presence  requirement  and  strategic  readiness 
requirement.28 

MR  is  a  Long  and  Complex  Process 

Even  though  these  methodologies  are  applied  to  only  one  or  a  few  scenarios,  they  require  a  significant 
amount  of  time  and  resources.  In  1991,  the  requirements  process  for  one  family  of  Naval  weapons 
needed  13  models,  at  least  four  different  working  groups  (one  to  three  star)  and  10-12  months  to  be 
completed.29  The  Army  needed  over  a  year  to  obtain  estimates  for  a  single  theater  using  its  detailed 
combat  model.30  The  Air  Force  and  Marines  generally  need  a  year  to  complete  their  processes.31 
The  cost  of  these  processes  in  terms  of  dollars  and  time  is  relatively  small  compared  to  the  cost  of  the 
munitions  themselves,  but  may  be  quite  large  when  compared  to  the  usefulness  and  clarity  of  the 
results. 

Outdated  and  Inconsistent  Modeling  Assumptions 

The  models  used  by  the  services  characterize  scenarios  with  a  variety  of  assumptions  that  directly 
affect  the  resulting  requirement.  RAND  says  that  the  data  used  by  the  services  is,  and  by  nature 
always  will  be  “very  soft. ”32  In  2000,  the  DoD  Inspector  General  criticized  the  services  because  they 
did  not  validate  these  assumptions,  and  when  asked,  could  not  produce  any  form  of  documentation 
to  support  the  numbers  used.33  As  examples  of  such  assumptions,  the  report  cites  the  rate  of  repair  of 
enemy  targets,  the  percentage  of  targets  that  are  false  (one  service  used  80%),  and  the  percentage  of 
friendly  forces  that  are  lost  in  combat.34  In  2001,  the  GAO  criticized  the  services  for  using  old 
munitions  effectiveness  data  that  had  not  been  updated  to  reflect  the  improved  accuracy  of  precision- 
guided  munitions.35  According  to  the  GAO,  the  services  continued  to  use  models  that  included 
assumptions  dating  back  to  the  Cold  War  to  calculate  requirements  for  billions  of  dollars  worth  of 
deep  attack  weapons  —  even  after  studies  by  the  Air  Force,  Navy,  and  two  DoD  reviews  identified  the 
problem.  The  purchase  of  these  deep  attack  weapons  would  effectively  double  the  nations  stock  of 
precision  weapons.36  Fundamental  changes  in  force  structure,  technology,  weapon  use  and  selection, 


27  OIG  p.  6 

28  At  the  time,  the  current  operations/forward  presence  requirement  was  called  the  residual  readiness  requirement. 

29  Kassing  et  al  p.  1 

30  Ibid  p. 

31  Ibid.  p.  31 

32  Kassing  et  al  p.33 

33  OIG  p.  7 

34  Ibid 
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and  overall  strategy / doctrine  since  the  fall  of  Russia  are  such  that  the  Defense  Science  Board  and  the 
GAO  both  have  serious  doubts  about  the  ability  of  current  models  to  depict  modern  warfare.37 

Sensitivity  to  Assumptions 

The  models  are  very  sensitive  to  their  assumptions.  This  leads  to  highly  variable  results.  RAND  calls 
them  ‘instable’  —  “Requirements  can  shift  drastically  throughout  whole  families  of  munitions  plans 
when  a  single  parameter  is  changed.  As  a  result  policymakers  can  lose  confidence  in  the  entire 
requirements  estimation  process.”38  The  DoD  Inspector  General  calls  the  process  ‘ad  hoc.’39 
Furthermore,  the  inputs  to  the  models  will  change  anytime  there  is  a  change  in  the  Defense  Planning 
Guidance,  Illustrative  Planning  Scenarios,  Threat  Report,  or  Phased  Threat  Distributions.  According 
to  the  GAO,  “these  process  components  significantly  affect  the  numbers  and  types  of  munitions 
needed  to  meet  the  Commander  in  Chiefs  objectives.”40  The  DoD  IG  says  the  same  when 
discussing  the  analytic  procedures  used  in  conjunction  with  the  models;  “the  analytic  procedures  have 
a  direct  and  significant  impact  on  the  final  calculation  of  requirements.”41 


Summary:  How  Do  We  Interpret  These  Critiques? 

The  criticisms  cited  here  outline  one  major  reason  why  it  has  been  difficult  for  munitions  planners  to 
make  the  transition  to  capabilities-based  planning  —  uncertainty  is  not  dealt  with  adequately.  The 
MR  process  attempts  to  predict  what  will  occur  in  the  future,  and  planners  add  margins  of  safety  at 
each  step  because  predictions  are  not  enough  to  ensure  preparedness  in  the  event  of  unforeseen 
events.  The  output  of  the  requirements  process  is  a  point  estimate  -  the  actual  number  of  each 
munition  required  for  the  armed  services  to  carry  out  their  mission.  That  number  is  then  used  as  a 
benchmark  and  justification  in  an  entirely  different  set  of  budgetary  processes  that  determine  how 
many  of  each  type  of  munition  will  be  ordered  and  produced.  At  this  point  another  problem  surfaces 
that  is  not  dealt  with  explicitly  in  the  above  documents,  but  that  is  nonetheless  widely  acknowledged 
within  munitions  planning  circles. 

There  is  no  direct  link  between  the  requirements  and  programming/budgeting  processes.  The  point 
estimate  output  of  the  requirements  process  is  not  very  useful  to  programmers  and  budgeters  as  they 
try  to  weigh  the  costs  and  benefits  of  a  munitions  portfolio  against  the  overall  budget  situation.  In 
today’s  environment  of  shrinking  funding,  the  process  does  not  include  integral  cost  considerations, 
nor  does  it  appear  to  provide  policymakers  an  adequate  understanding  of  how  flexibility  in  terms  of 
substitute  tactics,  weaponry,  or  contracts  might  affect  the  answer.  Policymakers  have  only  a  limited 
amount  of  time  during  the  budgeting  process  to  make  decisions  affecting  the  investment  of  billions 
of  dollars  and  future  wartime  performance.  The  current  process  provides  only  a  point  estimate 
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requirement.  There  is  really  no  way  to  assess  the  tradeoffs  in  cost  and  performance  of  a  variety  of 
possible  portfolios.  It  is  this  fact  that  has  drawn  the  ire  of  many  critics.  To  date,  documented  reports 
tend  to  attack  the  MR  process  itself  and  only  touch  upon  its  tenuous  link  to  the  budgeting  and 
programming  processes. 


CHAPTER  THREE 

Towards  a  Capabilities- based  Approach _ 

Towards  A  Capabilities-based  Approach 

Munitions  planners  face  a  resource  allocation  problem.  The  most  basic  question  has  always  been, 
“how  many  munitions  of  what  type  should  we  buy  in  order  to  meet  future  threats?”  The  question 
has  changed  dramatically  since  the  MR  process  was  created.  The  DoD  now  has  much  less  certainty 
about  the  exact  nature  of  the  future  threat  and  the  likelihood  of  receiving  full  funding  for  all  of  its 
projects.  A  capabilities-based  phrasing  of  the  same  question  is  “how  many  munitions  of  what  type 
should  we  buy  in  order  to  meet  future  threats,  given  that  we  are  not  exactly  sure  what  they  will  be  or 
the  amount  of  money  we  will  have  to  spend?”  This  phrasing  reveals  three  structural  handicaps  of  the 
current  process  that  must  be  improved  if  the  DoD  wishes  to  move  to  a  capabilities-based  system. 
Namely,  the  MR  process  (and  its  linkage  with  the  budgeting/programming  process)  does  not  take 
cost  into  account  in  an  appropriate  manner,  it  does  not  explicitly  deal  with  uncertainty,  and  does  not 
take  steps  to  improve  flexibility  by  considering  industrial  policies  along  with  procurement  decisions. 
This  chapter  explains  why  it  is  important  to  make  improvements  in  each  of  these  areas  if  the  DoD 
wishes  to  implement  a  capabilities-based  approach. 

Importance  of  Dealing  With  Cost  Adequately 

The  current  decision  process  does  not  include  cost  as  an  integral  factor  in  the  resource  allocation 
models.43  The  process  is  stepped  —  only  after  the  requirement  is  determined  is  it  constrained  by  cost. 
The  goal  of  the  requirements  office(s)  within  each  service  is  to  win  the  hypothetical  war  in  the  ‘best’ 
manner.  The  point  estimate  result  from  the  requirements  models  is  then  forwarded  to  programmers 
and  budgeters  who  match  it  to  the  budget. 

The  point  estimate  found  by  the  service  models  is  the  ‘best’  solution,  independent  of  cost.  Often 
times  it  is  too  expensive,  and  the  requirement  must  be  cut  or  modified  in  some  way  by  the 
programmers.  The  programmers  are  under  pressure  from  the  budget  cycle  and  do  not  have  the 
requirements  office’s  intricate  understanding  of  operational  tradeoffs  between  weapons.  The 
requirements  models  take  a  long  time  to  calibrate  and  run.  This  makes  it  hard  for  the  programmers 
to  give  feedback  and  receive  further  guidance.  As  a  result,  the  transformation  from  requirement  to 
budget  request  tends  to  be  ad  hoc. 

A  concrete  example  might  be  useful  here.  Last  year  an  Army  general  responsible  for  the  procurement 
budget  had  a  decision  to  make.  He  was  told  that  the  requirement  for  155mm  precision-guided 


42  DoD  IG  D-2000-079 

43  In  fact,  the  models  used  do  not  appear  to  be  resource  allocation  models  in  the  traditional  sense.  The  constraints  appear  to  be  the 
platforms,  doctrines,  and  tactics  presumed  to  be  available  at  the  time  of  the  war  (up  to  7  years  in  the  future). 
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artillery  was  120,000  rounds  —  at  $1 10,00  apiece!  That  is  about  $13.5  billion  worth  of  artillery, 
compared  to  an  Army  wide  annual  procurement  budget  of  $10.5  billion.  The  general  knew  that  he 
could  not  afford  the  entire  requirement  and  also  knew  that  there  must  be  some  mix  of  precision  and 
conventional  artillery  appropriate  to  the  Army’s  procurement  budget  and  goals.  But  he  could  not  get 
a  quick  analysis.  He  was  unable  to  receive  a  quick  and  clear  analysis  of  the  tradeoffs  between  cost  and 
performance  of  various  weapon  mixes.  In  the  end,  the  general  bought  less  than  75%  of  the 
requirement  because  that’s  what  he  thought  he  could  afford  —  and  noted  the  problem. 

The  key  here  is  to  understand  that  the  ‘best’  solution  changes  depending  on  the  amount  of  money 
available  and  that  receiving  only  part  of  an  unconstrained  solution  can  be  far  worse  than  receiving  the 
entirety  of  a  cost  constrained  solution.  One  high-ranking  Army  official  puts  it  this  way: 

It’s  like  calling  a  store  and  asking  for  the  best  possible  computer.  It  costs  $10,000,  but  you  only  have  an 
$8,000  line  of  credit,  so  they  send  you  the  monitor  this  year  and  the  CPU  next  year.  If  you  had  known 
what  you  had  to  spend,  what  everything  cost  and  were  there  to  make  the  decision,  you  probably  would 
have  gotten  a  cheaper  monitor  and  taken  the  whole  thing  now.  After  all,  the  monitor  doesn’t  affect  your 
performance  much  -  as  long  as  you  have  one. 

What  he’s  saying  is  that  there  are  marginal  dollar  tradeoffs  between  potential  investments  (munitions 
in  this  case)  that  greatly  affect  what  should  be  bought  at  each  level  of  funding.  Because  the  two 
groups  involved  in  the  process  have  different  problems  to  solve  (win  the  war,  and  fit  the  programs  to 
the  budget),  it  is  difficult  to  combine  their  efforts  and  take  such  tradeoffs  into  account.  The  Joint 
staff  of  the  DoD  is  currently  looking  for  ways  to  improve  the  relationship  between  these  groups  in 
order  to  become  more  aware  of  these  tradeoffs. 

Importance  of  Dealing  with  Uncertainty  Explicitly 

The  previous  chapter  showed  how  each  step  of  the  MR  process  makes  assumptions  in  order  to  pare 
an  inherently  large  degree  of  uncertainty  down  to  a  manageable  size.  Such  assumptions  include  the 
goals,  size,  type  and  duration  of  wars,  the  strategies  and  tactics  employed,  available  equipment,  and 
weapon  choice  and  effectiveness.  All  of  these  factors  have  a  direct  impact  on  the  number  and  type  of 
munitions  demanded  by  the  war  fighter.  The  inclusion  of  so  many  assumptions  may  make  the 
problem  easier  to  solve,  but  undermines  the  goal  of  a  capabilities-based  approach.  CBP  takes  for 
granted  that  uncertainty  is  a  fundamental  part  of  the  planning  problem;  that  the  Department  must 
be  prepared  for  a  variety  of  contingencies  and  that  such  preparation  must  take  place  within  some 
understood  economic  framework.  In  the  2001  Quadrennial  Defense  Review,  the  Bush 
administration  announced  a  shift  away  from  the  traditional  two  major  war  planning  requirement  to  a 
capabilities-based  planning  paradigm.^5 


44  Davis,  Paul.  “Analytic  Architecture  for  Capabilities-Based  Planning,  Mission-System  Analysis,  and  Transformation”  RAND, 
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Unfortunately,  the  Munitions  requirements  process  has  changed  little  since  that  time.  It  is  no 
wonder,  because  the  MR  process  was  created  to  answer  a  fundamentally  different  question.  That 
question  might  be  phrased  as  “given  that  I  must  be  able  to  win  this  particular  war  and  will  likely  be 
given  all  the  resources  I  need,  what  should  I  ask  for?”  The  new  question  is,  “given  that  I  am  not 
exactly  sure  what  my  mission  will  be,  how  do  I  best  prepare  with  the  amount  of  money  that  I  think 
will  be  available?”  The  sorts  of  decisions  that  must  be  made  are  fundamentally  different  -  policy 
makers  now  face  the  problem  of  cost  constrained  resource  allocation  under  uncertainty.  The  old 
system  is  simply  not  set  up  to  deliver  the  kind  of  information  needed  —  specifically,  an  understanding 
of  the  tradeoffs  between  various  strategies  across  a  multitude  of  budgetary  and  threat  scenarios. 

It  is  worth  noting  here  that  methods  used  for  planning  under  uncertainty  require  the  same  level  of 
concreteness  as  any  other  planning  method.  At  some  point,  strategies  must  be  weighed  against 
concrete  scenarios.  Capabilities-based  planning  as  described  by  Davis  is  a  mission-based  approach 
that  focuses  on  being  prepared  to  achieve  a  variety  of  goals,  rather  than  being  prepared  to  win  one  or 
two  (or  more)  regionally  specific  wars.46  The  underlying  character  of  the  uncertainty  must  be 
described  adequately  for  such  methodologies  to  be  useful.  The  MR  process  does  not  explicitly 
consider  uncertainty  —  with  the  results  detailed  in  previous  sections.  In  later  sections,  this  paper  will 
characterize  uncertainty  as  an  uncertainty  about  target  types  and  the  length  of  time  we  have  to 
prepare  for  a  war.  We  will  provide  some  discussion  about  the  uncertainty  of  war  size. 

Importance  of  Considering  Industrial  Policies  Concurrently  With  Procurement 

Once  one  considers  uncertainty  and  budget  explicitly,  strategies  that  can  improve  flexibility  across  a 
variety  of  contingencies  suddenly  become  feasible  and  potentially  very  useful.  In  this  case,  such 
strategies  might  include  spending  some  portion  of  money  on  industrial  policies  that  can  act  as 
options  in  the  event  of  a  surprise  war  —  keeping  extra  lines  available  for  surge  production  or  keeping 
long  lead  components  in  stock  ‘just  in  case.’  The  current  requirements  and  budgeting  processes 
process  do  not  consider  industrial  policies  as  investments  that  can  lead  to  strategic  flexibility  and 
improved  wartime  capability.  Policymakers  want  to  know  how  many  of  each  type  of  weapon  should 
be  bought  and  stored.  But  as  weapon  costs  and  mission/demand  uncertainties  increase,  it  may  be 
better  to  invest  in  production  as  well  as  in  (or  in  some  cases  in  lieu  of)  stock. 

The  DoD  may  be  able  to  improve  overall  system  performance  by  making  calculated  investments  in 
PGM  production  technologies  and  processes  when  it  cannot  afford  to  invest  as  heavily  in  physical 
weaponry.  Traditionally,  munitions  have  been  treated  as  a  ‘safety  stock’  item.  Because  demand  for 
munitions  is  erratic,  and  the  penalty  for  not  having  enough  is  so  high,  the  DoD  has,  in  the  past, 
stocked  ‘a  lot  of  everything.’  That  is,  it  has  invested  heavily  in  its  actual  physical  stock  of  munitions. 
PGMs  are  more  expensive  than  unguided  munitions,  and  the  DoD  cannot  afford  to  stock  them  in 
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the  same  manner.  In  some  cases,  the  cost  of  a  guided  weapon  program  is  comparable  to  the  cost  of  a 
platform  system.47 

At  the  same  time,  the  DoD  finds  itself  facing  a  wider  variety  of  potential  missions  —  each  of  which 
likely  has  a  different  ‘optimal’  mix  of  weaponry.  Budget  constraints  may  force  the  department  to 
prioritize  preparedness  between  missions.  The  inclusion  of  industrial  policies  in  the  planning  process 
has  the  potential  to  increase  flexibility  and  performance  across  all  missions.  Specific  investments  in 
surge  capacity  and  options  in  the  form  of  stored  components  or  shared  lines  can  decrease  the  DoD’s 
yearly  procurement  costs  for  certain  weapons/ contingencies.  This  would  free  up  money  to  be  spent 
on  physical  weapons  that  provide  maximum  benefit.  Obviously,  such  decisions  must  be  weighed 
carefully  to  ensure  that  the  right  missions  have  priority  and  that  there  is  enough  time  to  surge 
production  or  call  in  options.48  Recent  replenishment  concerns  suggest  that  making  these  decisions 
is  worthwhile  -  recent  conflicts  have  been  characterized  by  post  conflict  rushes  to  reconstitute  stock. 
Planning  in  advance  for  replenishment  might  make  it  cheaper,  more  efficient  and  less  frenetic. 

It  appears  that  current  production  rates/characteristics  are  the  result  of  compromises  made  by 
programmers  and  budgeters  as  they  match  the  yearly  budget  to  the  requirement  -rather  than  a 
premeditated  decision  process.  The  munitions  industry  sees  demand,  not  in  terms  of  targets  and 
wars,  but  in  terms  of  contracts  and  appropriations.  It  is  shielded  from  demand  uncertainties  by  the 
requirements  process.  Contracts  are  structured  individually  so  as  to  deliver  the  required  munitions 
over  time  within  the  appropriated  budget.  If  an  expensive  weapon  is  deemed  necessary,  programmers 
will  stretch  its  cost  over  a  number  of  years  to  make  it  affordable.  The  prime  contractor  comes  up 
with  a  design,  finds  suppliers,  and  builds  an  assembly  line  to  fit  that  schedule.  The  TLAM  might  be 
a  good  example  of  this  —  concerns  have  arisen  over  replenishment  multiple  times.  The  weapon  is 
expensive,  at  almost  $  1  million  apiece.  The  assembly  lines  are  set  up  to  produce  them  at  a  rate  of  40 
per  month,  with  a  lead-time  of  12  months.  About  700  Tomahawks  were  fired  during  the  first  week 
of  Operation  Iraqi  Freedom.49  It  will  take  2  _  years  to  replenish  just  one  week’s  worth  of 
Tomahawks.  At  the  same  time,  the  administration  had  no  trouble  raising  $80  billion  in  emergency 
appropriations  to  cover  the  costs  of  the  war.  The  DoD  may  be  able  to  increase  the  effectiveness  of  its 
yearly  budget  by  considering  production  concurrently  with  procurement. 

This  paper  will  not  explore  the  effectiveness  of  the  surge  production  and  option  policies.  Initial 
modeling  efforts  showed  that  to  model  these  policies  would  be  too  complex  for  the  purposes  of  this 
illustrative  analysis  (showing  that  exploratory  analysis  and  robust  decision  making  techniques  can  be 
usefully  applied  to  the  MR  problem).  This  paper  will  explore  production  policy  as  the  number  of 
production  lines  of  each  weapon.  The  number  of  production  lines  for  a  weapon  will  operate  as  the 
‘production  constraint,’  limiting  the  number  of  weapons  available  for  a  conflict.  The  number  of 
weapons  available  for  a  conflict  will  also  be  affected  by  the  amount  of  time  we  have  to  prepare  for  a 


47  The  Tomahawk  program  cost  is  $14  B,  compared  to  $10B  lor  the  Harrier,  $8.4  B  for  the  JSTARs,  and  $2.6B  for  the  F-l  17a. 
Source:  CDI,  “Military  Almanac:  2001-2002” 
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0  Peters  et  al  (2004)  have  shown  that  many  recent  contingencies  have  offered  a  significant  amount  of  strategic  warning  time. 
Source:  Global  Security.org/military/ops/Iraqi_freedom-numbers.htm 
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war.  We  will  find  that  considering  production  does  indeed  affect  the  viability  and  flexibility  of  a 
munitions  portfolio. 


How  This  Dissertation  Will  Address  Cost,  Production  and  Uncertainty 

Cost,  production  and  uncertainty  should  be  considered  concurrently  if  we  wish  to  pursue  a 
capabilities-based  approach  to  munitions  planning.  Later  chapters  will  present  how  this  dissertation 
will  examine  cost  and  production  capacity  as  constraints  and  preparation  time  and  war  scenario  as 
uncertainties.  First  we  will  vary  budget,  production  capacity  and  preparation  time  parametrically  for 
a  single  scenario  using  the  methodology  of  exploratory  modeling/ analysis.  This  exploratory  analysis 
will  demonstrate  how  much  each  factor  changes  the  optimal  munitions  portfolio  and  show  that  this 
information  can  easily  be  provided  to  a  policymaker  for  consideration.  We  will  then  show  the 
usefulness  of  searching  for  a  solution  that  is  robust  across  a  variety  of  uncertain  scenarios.  Finally,  we 
will  show  that  the  search  for  a  robust  solution  may  be  conducted  across  scenarios,  budget  levels, 
production  characteristics  and  preparation  times  in  order  to  create  flexible,  affordable,  and  effective 
munitions  portfolios.  In  the  meantime,  the  next  chapter  surveys  other  methodologies  and  explains 
why  they  are  not  as  well  suited  to  this  problem. 


CHAPTER  FOUR 

Overview  of  Relevant  Literature 


Relevant  Literature 

The  problem  faced  by  the  DoD  is  a  complex  planning  problem  that  involves  resource  allocation  and 
inventory  control  under  uncertainty.  Planners  must  divide  a  limited  number  of  resources  (budget) 
across  a  variety  of  investments  (munitions),  the  value  of  which  depends  on  what  actually  happens  in 
the  future  (war).  Resource  allocation  and  inventory  control  are  the  two  most  relevant  academic 
literatures.  We  shall  see  that  the  character  of  the  uncertainties  faced  by  munitions  planners  is 
precisely  what  makes  the  problem  so  complex,  and  what  set  it  apart  from  the  problems  typically  dealt 
with  in  the  two  literatures. 


Resource  Allocation 

Resource  allocation  models  will  not  work  for  this  problem  because  the  uncertainty  faced  by  war 
planners  cannot  be  reduced  to  a  probability  distribution.  At  least  three  entire  fields  —  economics, 
finance  and  operations  research  -  are  devoted  to  understanding  and  solving  resource  allocation 
problems.  Methodologies  include  linear  programming,  optimization,  portfolio  techniques,  profit 
maximization,  cost  minimization,  and  a  variety  of  modeling  and  simulation  techniques.  The 
problem  with  simply  applying  one  or  more  of  these  techniques  alone  to  PGM  selection  is  that  we  face 
a  wide  variety  of  possible  futures,  and  these  models  can  only  capture  one  at  a  time.  It  is  possible  to 
model  multiple  futures  —if  one  has  a  clear  idea  of  the  probability  of  each  futures  occurrence  and  of 
one’s  own  preferences.  In  this  case,  once  could  create  a  model  that  maximizes  expected  utility,  or 
expected  return  across  the  ensemble  of  futures.  But  is  it  really  possible  to  place  a  probability  on  the 
occurrence  of  each  war?  Is  it  even  possible  to  conceive  of  all  the  wars  that  may  happen? 

The  events  that  war  planners  contend  with  are  fundamentally  not  probabilistic.  But  the  services  use 
resource  allocation  methodologies  now,  with  a  wide  variety  of  simplifying  assumptions  aimed  at 
getting  around  the  problem  of  uncertainty.  The  USDP  actually  prescribes  the  wars  that  will  occur, 
giving  each  war  a  de  facto  probability  of  %100.  From  that  first  assumption  stem  many  more  —  the 
number  of  targets  by  type,  the  amount  of  time  we  have  to  prepare,  our  tactics  etc.  Unfortunately,  it 
is  not  clear  in  the  resulting  point-estimate  which  or  how  many  of  these  assumptions  have  been  made, 
or  how  a  change  in  them  will  affect  the  performance  of  the  recommended  portfolio.  Resource 
allocation  methodologies  are  incredibly  useful  to  the  DoD,  but  must  be  placed  within  the  context  of 
multiple  missions  and  uncertainty  about  the  likelihood  of  each  occurring. 
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Inventory  Control 

The  field  of  inventory  control  is  a  subset  of  operations  research  and  uses  many  of  the  same 
methodologies.  The  goal  of  inventory  control  is  to  find  the  appropriate  balance  between  stock  and 
productive  capacity  to  ensure  that,  on  the  one  hand,  resources  are  not  being  wasted  on  unneeded 
stock,  and  on  the  other  hand,  that  there  is  always  enough  stock  to  meet  demand.50  Dynamic 
programming,  linear  programming,  Markovian  processes  and  simulation  figure  prominently  in  the 
literature.  However,  each  methodology  requires  an  estimation  of  demand,  usually  in  the  form  of  a 
probability  distribution.  ‘True’  demand  for  munitions  is  virtually  impossible  to  predict  or  fit  to  a 
distribution  in  advance.  It  depends  on  many  factors,  such  as  the  size  duration  and  goal  of  a  military 
action,  as  well  as  tactics,  equipment,  terrain  and  weather.  Furthermore,  the  nature  of  combat 
operations  results  in  long  periods  of  low  demand  (peacetime  training)  followed  by  huge  spikes  during 
wartime.  There  is  a  documented  hole  in  the  literature  surrounding  the  issue  of  erratic  or  ‘lumpy’ 
demand.51 

As  we  have  discussed,  the  services  have  dealt  with  the  problem  of  erratic  demand  by  stocking  more 
than  is  needed  -  by  holding  a  large  ‘safety  stock.’  A  safety  stock  is  the  inventory  held  above  ‘normal’ 
to  serve  as  a  buffer  against  production  problems  and  demand  swings.  Indeed,  the  ad  hoc 
modifications  made  to  the  MR  process  today  appear  to  speak  to  this  strategy.  Unfortunately,  unless 
something  changes,  the  sheer  cost  of  PGMs  precludes  the  use  of  excessive  safety  stocks.  However,  the 
literature  supports  the  notion  that  changes  in  the  production  process  can  positively  affect  the  resource 
allocation  problem  by  reducing  the  required  safety  stock.  Papers  by  David  A  Collier  and  Kenneth  R 
Baker  show  that  increases  in  component  commonality  decrease  the  amount  of  safety  stock  required.52 
Donald  Gross  and  A.  Soriano  show  that  decreases  in  manufacturers  lead  times  decrease  the  amount 
of  safety  stock  required.53 

This  paper  will  not  deal  explicitly  with  lead-time  and  component  commonality  issues.  Our  primary 
emphasis  is  in  demonstrating  a  methodology  that  allows  munitions  planners  to  implement  the  ideas 
of  capability  based  planning.  This  paper  will  show  that  by  using  exploratory  modeling  and  robust 
analysis,  we  may  choose  our  safety  stock  wisely  even  when  facing  budget  constraints  and  an  uncertain 
future.  That  is,  we  will  be  able  to  see  which  weapons  should  be  stocked  for  each  level  of  budget, 
production  and  preparation  time  first  for  a  single  war,  and  second  across  a  wide  variety  of  potential 
wars. 


50  Supervision  of  the  supply,  storage,  and  accessibility  of  items  in  order  to  insure  an  adequate  supply  without  excessive  oversupply. 
Hyperdictionary  online 

51  Silver,  Edward  A.  “Operations  Research  in  Inventory  Management:  A  Review  and  Critique”  Operation  Research,  Vol.29,  No4 
1981:  p.640 

J  Collier,  David  A.  “Aggregate  Safety  Stock  Levels  and  Component  Part  Commonality”  Management  Science,  Vol.  28,  No.l  1, 
1982  ;  Baker,  Kenneth  et  al  “The  Effect  of  Commonality  on  Safety  Stock  in  a  Simple  Inventory  Model”  Management  Science,  Vol.  32, 
No.  8,  1986 

o 

JD  Gross,  Donald  and  Soriano,  A.  “The  Effect  of  Reducing  Lead  Time  on  Inventory  Levels  —  Simulation  Analysis” 
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Conclusions 

Pure  resource  allocation  models  are  impractical  because  it  is  impossible  to  fit  the  uncertainty 
associated  with  the  occurrence  and  nature  of  a  war  to  a  probability  distribution.  Pure  inventory 
control  models  are  impractical  for  the  same  reason  -  because  demand  for  munitions  is  uncertain  and 
erratic.  Although  we  cannot  find  the  ‘optimal’  solution,  we  do  have  tools  available  now  that  can 
provide  better  answers  than  the  ones  we  have  now.  These  methods,  detailed  in  the  next  section, 
enable  policymakers  to  understand  the  economic  tradeoffs  made  with  various  resource  allocation 
strategies  across  a  wide  variety  of  constraints  and  war  scenarios.  In  “Research  Portfolio  for  Inventory 
Management  and  Production  Planning”  Harvey  M.  Wagner  hints  at  a  methodology  similar  to  the 
one  that  will  be  proposed  in  the  next  chapter: 

Most  planning  models  do  not  attempt  intrinsically  to  account  for  uncertainties,  such  as  the  range  of 
variability  in  future  demand.  If  the  analyst  can  identify  a  few  parameters,  each  of  which  possibly  has 
several  different  values,  then  a  set  of  case  studies  can  be  run  to  investigate  the  differential  impact.  For 
example,  a  base  case  can  be  constructed  and  compared  with  optimistic  and  pessimistic  scenarios.  Still, 
rarely  are  the  results  of  a  planning  model  analysis  couched  as  a  contingency  plan  reflecting  future 
uncertainties.  Yet,  in  real  situations,  the  results  of  planning  analysis  almost  always  are  revised  as  the 
uncertainties  are  resolved.  Currently,  companies  use  ad  hoc  procedures  to  alter  plans  to  reflect  realized 
deviations  in  model  input  assumptions.  Operations  researchers  ought  to  examine  effective  processes  for 
transforming  planning  model  solutions  into  contingency  rules. 54 

Wagner  tells  us  that  it  is  actually  quite  common  for  organizations  to  gloss  over  uncertainties  during 
model  creation,  make  ad  hoc  changes  to  their  plans  to  account  for  it.  He  points  out  that  by 
identifying  an  uncertain  variable  and  exploring  the  effect  of  changing  its  value,  we  can  understand 
how  it  might  affect  our  plans.  We  can  learn  how  the  uncertainty  itself  shapes  the  strategy  we  should 
choose. 


^  Wagner,  Harvey  M  “Research  Portfolio  for  Inventory  Management  and  Production  Planning”,  Operation  Research,  Vol  28,  No. 
3  1980:  p.457 
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Overview 

This  report  provides  an  alternative  capabilities-based  framework  for  munitions  planning  that  allows 
policymakers  to  see  and  understand  the  tradeoffs  involved  in  their  decisions,  taking  into  account  the 
uncertainties  and  constraints  specific  to  this  problem  —  namely  the  constraints  of  budget  and 
production  and  the  uncertainties  of  preparation  time  and  occurrence  of  war.  This  paper  proposes  the 
employment  of  exploratory  modeling  and  robust/ adaptive  planning  to  represent  the  budget, 
production  constraints,  preparation  time  and  war  in  such  a  way  that  planners  can  see  potential 
sources  of  flexibility  (and  limitation)  in  precision  guided  munitions  planning.  By  using  these 
methods,  we  will  avoid  making  many  premature  assumptions  about  the  future  we  face  and  will  be 
better  able  to  create  strategies  that  perform  well  across  a  variety  of  futures  while  simultaneously 
operating  within  an  economic  framework. 


A  Resource  Allocation/Inventory  Management  Problem? 

Munitions  procurement  can  be  thought  of  as  a  straightforward  resource  allocation  problem, 
combined  with  inventory  management.  There  is  a  budget.  There  is  a  cost  to  production, 
procurement  and  storage.  There  is  some  value  associated  with  having  a  particular  mix  of  weaponry 
during  a  war.  There  is  also  some  penalty  associated  with  having  a  shortage.  Each  weapon  also  has  a 
set  of  production  characteristics  that  may  improve  or  deteriorate  its  availability.  What  makes 
munitions  procurement  different  from  traditional  problems  of  this  sort  are  the  number  and  type  of 
uncertainties  with  which  the  buyer  contends.  We  know  what  it  costs  to  buy,  store,  and  dispose  of 
weapons.  We  also  know  how  long  it  takes  to  build  weapons.  But  we  can’t  make  a  resource 
allocation  or  inventory  control  model  because  we  don ’t  know  aggregate  demand55  —  the  size,  location 
and  politics  of  war),  enemy  targets  and  tactics,  value  of  the  war,  penalty  for  a  shortage,  amount  of 
warning,  or  probability  of  occurrence.  The  vast  majority  of  these  variables  directly  affect  demand.  If 
these  variables  were  known  with  certainty,  it  would  be  possible  to  calculate  an  optimal,  cost-effective 
portfolio  of  weaponry  and  the  productive  capacity  needed  to  build  it  in  time.  The  MR  process 
systematically  generates  a  point  estimate  for  each  uncertainty.  This  simplifies  the  problem,  but  at  the 
expense  of  understanding  potential  tradeoffs.  If  it  is  assumed  that  only  one  scenario  will  occur,  we 
have  a  set  number  of  years  to  prepare,  and  that  cost  is  no  object,  then  the  answer  for  that  scenario  can 
be  calculated,  and  there  is  no  need  to  consider  tradeoffs. 

A  traditional  resource  allocation  model  will  not  solve  this  problem  because  we  cannot  put  a 
probability  on  the  occurrence  of  any  single  war.  A  traditional  inventory  model  will  not  solve  this 
problem  for  the  same  reason  —  we  don’t  know  what  demand  will  be,  cannot  predict  it,  and  cannot  fit 


55  Peters,  John  et.  al.  “Satisfying  Demand  for  Selected  Munitions :  a  Production  Enhancement  Study”  RAND,  2003. 
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it  to  a  distribution.  However,  we  can  use  a  resource  allocation  model  combined  with  inventory 
policies  as  the  foundation  for  an  exploratory  model  that  would  allow  us  to  understand  how  those 
uncertainties  affect  a  strategy.  Such  an  approach  will  allow  us  to  vary  the  uncertainties  affecting 
demand  and  create  a  strategy  that  works  fairly  well  across  multiple  scenarios.  Furthermore,  this 
approach  will  enable  us  to  understand  why  a  particular  strategy  works,  what  its  limitations  are,  and 
what  the  tradeoffs  are  between  scenarios  and  strategies. 


What  is  Exploratory  Analysis? 

‘Traditional’  policy  analysis  has  tended  to  attack  problems  of  deep  uncertainty  by  narrowing  down 
the  possible  futures  to  those  that  are  most  likely  to  occur  and  devising  the  most  optimal  strategy  for 
each  future.  The  traditional  approach  asks  first,  ‘what  is  most  likely  to  occur?,’  creates  a  scenario,  and 
then  finds  the  most  optimal  solution  to  that  particular  scenario.  Such  planning  tools  have,  at  times, 
worked  well  in  the  past,  and  may  be  useful  in  dealing  with  problems  that  have  very  few  key 
uncertainties  and / or  uncertainties  that  create  futures  with  only  slight  differences.  However,  strategies 
based  on  a  ‘prediction’  of  the  future  can  fail  spectacularly  if  that  future  fails  to  materialize.  Take,  for 
example,  a  fund  manager  who  in  1994,  believed  that  there  would  be  a  bear  market  for  the  next  five 
years  —  or  a  fund  manager  who  in  1998  believed  the  market  would  continue  to  rise  through  2003. 

Exploratory  analysis,  on  the  other  hand,  refrains  from  making  assumptions  about  the  future  and  is 
the  first  part  of  a  methodology  that  asks,  ‘what  strategy  will  deal  best  with  the  uncertainty  we  face?’ 
To  be  more  specific,  an  exploratory  analysis  is  the  first  step  in  answering  this  question.  Such  an 
analysis  generates  a  range,  or  ‘landscape’  of  all  plausible  futures  in  order  to  understand  how 
uncertainty  can  affect  a  problem,  which  uncertainties  are  critical  to  a  problem,  and  when  such 
uncertainties  matter.  Robert  Lempert  and  George  Park  (1998)  put  it  best: 

One  reason  our  approach  is  useful  is  that  there  is  often  a  great  deal  of  information  about  a  problem  that, 
although  insufficient  for  making  accurate  predictions,  is  nonetheless  useful  for  making  decisions.  (...) 

While  it  may  seem  that  abandoning  a  best  estimate  for  a  large  set  of  plausible  futures  complicates  the 
decision-making  problem,  the  large  set  of  plausible  scenarios  often  represents  real  and  very  useful 
information.  Perhaps  surprisingly,  when  we  trade  the  question  What  is  most  likely  to  happen  in  the 
future?’  for  ‘Which  policy  choice  deals  best  with  the  uncertainties  we  face?’  the  complexity  posed  by  an 
unpredictable  future  often  falls  away  and  reveals  a  set  of  clear  choices.^ 

In  an  exploratory  analysis,  the  fund  managers  mentioned  above  would  relax  their  assumptions  and 
forecast  a  large  number  of  possible  futures  by  varying  the  interest  rate,  the  performances  of  various 
markets,  and  a  variety  other  important  factors.  They  would  then  examine  this  range  of  possible 
futures,  looking  for  relationships  in  the  key  uncertainties  that  would  change  the  nature  of  the 
problem  or  force  them  to  change  strategies. 


Park,  George  S.  and  Robert  J.  Lempert,  The  Class  of  2014  —  Preserving  Access  to  California  Higher  Education,  Santa  Monica 
(CA):  RAND  Education,  1998.  p.  5 
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Exploratory  analysis  is  a  distinctly  quantitative  tool.  It  is  therefore  helpful  to  recant  the  discussion  in 
modeling  terms.  Models  are  often  used  as  optimizing  tools  —  designed  to  find  the  optimal  value  of 
one  or  more  input  variables,  given  a  specified  objective  and  limited  by  constraints  in  other  input 
variables.  For  optimization  to  work,  all  aspects  of  the  model  other  than  the  parameters  of  interest 
must  be  defined  as  clearly  as  possible.  Therefore,  traditional  optimization  techniques  involve  the 
creation  of  best  estimate  current  or  most  likely  future  scenarios  that  are  built  into  the  model  in  the 
form  of  parameter  values  and  constraints.  Uncertainties  are  individually  identified  and  dealt  with 
before  the  model  is  run,  as  in  the  case  of  the  MR  Process.  Once  an  optimal  value  has  been  found,  a 
sensitivity  analysis  is  conducted  to  determine  how  much  the  ‘value’  of  the  objective  would  change  as  a 
result  of  variations  and/or  errors  in  the  inputs. 

An  exploratory  analysis  does  not  optimize  for  one  particular  scenario,  but  forecasts  the  value  of  the 
objective  function  over  a  whole  range  of  possible  scenarios.  The  model  is  run  many  times  -  each 
time  varying  inputs  and  constraints  to  create  a  landscape  of  possible  futures.  In  examining  this 
landscape,  a  skilled  analyst  comes  to  a  better  understanding  of  how  the  various  parameters  and 
uncertainties  surrounding  them  affect  the  problem. 

The  basic  idea  behind  exploratory  analysis  is  not  new,  but  until  recently  it  has  been  difficult  to  put 
into  practice  for  problems  of  more  than  just  a  few  variables  -it  becomes  impossible  for  the  human 
brain  to  conceptualize  uncertainty  beyond  a  few  factors.  Recent  improvements  in  computing  -  cheap 
memory,  fast,  networked  processes  and  powerful  visualization  tools  -  have  made  it  increasingly 
possible  to  apply  these  ideas  in  a  useful  way.  The  interpretation  of  an  exploratory  analysis  remains 
something  of  an  art  form.  The  analyst  must  find  ways  to  display  and  conceptualize  the  multiple 
dimensions  considered  mathematically  in  order  to  gain  useful  insight.  Those  researchers  conducting 
such  analyses  readily  admit  this  fact  and  remark  that  work  is  currently  being  done  to  improve  the 
ability  of  computers  to  mimic/improve  upon  human  abstraction  and  display  multidimensional 
information.  An  example  is  the  Computer  Assisted  Reasoning  system  (CARs)  developed  by  Evolving 
Logic,  which  allows  the  analyst  to  store  and  visually  represent  massive  amounts  of  multi  dimensional 
data  in  new  and  useful  ways. 

A  Pertinent  Example  —  ‘Weapon  Mix  and  Exploratory  Analysis’ 

“Weapon  Mix  and  Exploratory  Analysis,”  a  1997  RAND  report  by  Arthur  Brooks,  Steve  Bankes  and 
Bart  Bennett,  provides  an  excellent  example  of  exploratory  analysis  applied  to  a  problem  similar  to 
this  one.  The  body  of  the  report  is  devoted  to  highlighting  the  advantages  of  exploratory  analysis  to 
the  practicing  analyst.  The  study  demonstrates  the  difference  between  conventional  and  exploratory 
analysis  in  the  context  of  a  weapons  mix  problem.  The  problem  is  that  of  choosing  an  appropriate 
stockpile  of  weapons  for  a  specific  war  scenario.  In  this  respect  it  is  not  unlike  the  modeling  portion 
of  the  current  MR  process.  In  fact,  the  weapons  mix  study  even  uses  a  model  similar  to  the  one  used 
in  the  Air  Force’s  Deep  Attack  Weapons  Mix  Study  discussed  above.57  While  traditional 


57 


Borkks,  Arthur  et.  al.  “Weapon  Mix  and  Exploratory  Analysis:  A  Case  Study”  RAND.  Santa  Monica,  1997.  p.  6 
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optimization  yields  only  one  optimal58  result  for  the  scenario,  the  exploratory  analysis  found  196 
solutions  that  were  very  close  to  the  optimal.  They  were  close  to  optimal  in  the  sense  that  none  of 
the  new  solutions  increased  the  length  of  the  war  by  more  than  24  hours. 

The  presence  of  multiple,  near-optimal  solutions  gives  the  analyst  much  more  flexibility  in  analyzing 
performance  with  any  number  of  other  measures.  For  example,  once  the  optimal  weapon  bundles  are 
found,  the  analyst  can  look  at  the  cost  of  each  and  compare  that  cost  to  the  marginal  change  in 
performance.  The  analyst  can  also  look  at  how  the  various  bundles  fare  when  scenario  parameters 
change.  The  Weapon  Mix  Study,  for  example,  examines  the  performance  of  the  196  near-optimal 
bundles  under  varying  weapon  reliability  scenarios,  shows  which  bundles  are  ‘robust’  to  reliability 
changes,  and  explains  why.  Additionally,  one  could  compare  solutions  across  scenarios,  examining 
how  they  fare  in  each.  Traditional  analysis  takes  the  single  ‘most  optimal’  solution  as  a  given  and 
then  conducts  a  sensitivity  analysis  in  order  to  understand  its  limitations.  Exploratory  analysis  seeks 
to  understand  the  limitations  and  tradeoffs  of  multiple  near  optimal  solutions  at  once.  In  doing  so,  it 
provides  the  decision  maker  with  increased  flexibility  and  a  better  understanding  of  the  problem  and 
the  forces  that  shape  the  region  of  feasible  solutions. 

The  weapons  mix  study  is  important  because  it  provides  ample  evidence  for  the  effectiveness  of 
exploratory  analysis  in  dealing  with  this  particular  problem.  The  key  differences  between  that 
research  and  the  research  proposed  here  are  the  audience  and  the  scope  of  the  problem.  The 
audience  for  this  research  will  be  the  policymaker  rather  than  the  analyst.  The  work  will  focus  more 
on  the  strategic  choices  than  the  inter-theater  tactical  ones.  This  research  will  look  for  strategies  that 
work  well  across  multiple  war  scenarios,  rather  than  alternative  strategies  within  a  particular  scenario. 

How  Does  Exploratory  Analysis  Fit  into  the  Planning  Process? 

Problems  with  multiple  objectives,  challenges  and  alternative  strategies  require  an  assessment  and 
understanding  of  tradeoffs  across  multiple  dimensions  —  systems  analysis  looking  at  costs, 
effectiveness  versus  cost,  and  various  intangibles.  As  such,  Exploratory  Analysis  is  most  useful  in  the 
initial  stages  of  strategy  development.  It  is  best  used  in  situations  where  several  strategies  are  debated 
on  different  but  apparently  valid  terms.  Exploratory  Analysis  can  prevent  the  decision-maker/ analyst 
from  making  premature  decisions  about  what  constitute  the  key  factors  of  risk  or  most  likely 
circumstances.  Such  an  analysis  encourages  one  to  remain  open-minded  to  a  wide  variety  of 
alternative  solutions  to  a  particular  problem  that  may  intuitively  seem  unrealistic.59 


What  is  Robust/Adaptive  Planning? 

Robust /Adaptive  Planning  seeks  to  identify  strategies  that  perform  well  across  a  range  of  possible 
futures  —  i.e.  strategies  that  are  robust  to  the  key  uncertainties  of  a  problem.  It  is  the  second  part  of 


The  study’s  primary  performance  measure  is  time  to  complete  the  war. 

5^  Davis,  Paul  K.  and  Richard  Hillestad,  Exploratory  Analysis  for  Strategy  Problems  with  Massive  Uncertainty,  unpublished 
manuscript,  Santa  Monica  (CA):  RAND,  2000.  p.32 
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the  tool  that  asks,  ‘given  we  cannot  predict  the  future,  what  actions  should  we  take?’60  After 
examining  the  landscape  of  possible  futures  provided  by  the  exploratory  analysis,  the  analyst  creates 
strategies  and  simulates  their  performance  across  all  futures.  The  relative  performance  of  each 
strategy  may  be  measured  by  compiling  the  expected  regret  of  alternative  strategies  across  scenarios. 
Regret  is  the  difference  between  the  expected  performance  of  a  strategy  being  tested  on  a  scenario  and 
the  performance  of  the  optimal  strategy  under  perfect  information.61  Computer  search  routines  are 
used  to  find,  first,  strategies  whose  expected  regret  is  never  large  compared  to  the  alternatives,  and 
second,  key  tradeoffs  among  strategies  and  assumptions  that  drive  the  choice  among  strategies.62 

Such  searches  are  used  both  to  suggest  strategies  for  consideration  and  to  test  hypotheses  about 
strategies.  The  search  may  show  that  one  strategy  performs  well  against  one  set  of  scenarios,  and 
another  performs  well  for  a  different  set.  The  analyst  may  then  create  a  combination  of  the  two 
strategies  and  test  to  see  whether  the  combined  strategy  is  indeed  more  robust  across  the  entire  range 
of  scenarios.  In  this  manner,  the  situations  under  which  a  promising  strategy  fails  are  uncovered,  and 
the  analyst  is  able  to  further  shape  the  strategy  through  the  use  of  hedges  or  other  modifications. 
Often  times,  an  adaptive  or  sequential  strategy  that  allows  for  either  organic  or  planned  policy 
changes  to  occur  in  response  to  new  information  is  the  most  robust  strategy. 

Davis  and  Hillestad  identify  a  number  of  general  ways  to  address  risk  and  uncertainty  that  may  be 
considered  in  the  creation  of  a  strategy.63  They  suggest  ignoring  uncertainty  when  the  worst-case 
consequences  are  tolerable.  Risk  can  be  reduced  by,  first,  eliminating  particular  sources  of  risk  (for 
instance,  buying  out  the  competition,  or  blowing  up  a  bridge),  and  second,  improving  the  quality  of 
prediction.  Management  can  insure  against  problems  by  purchasing  insurance,  hedging  or  a  portfolio 
approach.  Finally,  uncertainty  can  be  dealt  with  by  increasing  adaptiveness,  pooling  of  resources  and 
making  decisions  sequentially. 

How  Does  Robust/ Adaptive  Planning  Fit  into  the  Planning  Process? 

Robust/Adaptive  Planning  is  best  used  after  an  exploratory  analysis  has  been  conducted.  As  the 
search  for  a  strategy  that  works  well  across  a  variety  of  futures,  it  is  perhaps  most  useful  to  decision 
makers  working  at  the  highest  level.  When  combined  with  an  Exploratory  Analysis,  it  is  indeed  a  very 
powerful  tool  that  forces  a  decision  maker  to  keep  his/her  options  open,  and  think  about  ways  in 
which  strategies  can  be  improved  in  certain  areas  without  being  weakened  in  others. 


66  Lempert,  Robert  J.  and  Michael  E.  Schlesinger,  Robust  Strategies  for  Abating  Climate  Change,  Climatic  Change,  45  (1),  pp.  387- 
401,  2000.  p.  391 

61  Ibid. 

62  Ibid  p.392 

63  Dais  and  Hillestad,  2000.  p.  30 


CHAPTER  SIX 

Setting  Up  the  Illustrative  Analysis 


Overview 

This  section  is  devoted  to  describing  the  models  that  will  be  used  in  our  illustrative  analysis.  In  order 
to  simultaneously  include  cost,  production  and  performance  under  uncertain  circumstances,  we 
address  the  following  two  questions: 

1.  How  do  the  constraints  of  budget  and  available  production  lines  and  the  uncertainty  of 
preparation  time  before  a  war  affect  the  optimal  munitions  portfolio  when  we  assume  we 
know  which  war  scenario  will  occur? 

2.  Given  that  we  do  not  know  which  war  will  occur,  how  do  we  find  a  portfolio  of  munitions 
that  works  well  across  a  variety  of  scenarios,  taking  into  account  the  aforementioned 
uncertainty  and  constraints? 

Our  policy  lever  is  the  reserve  portfolio  —  a  planned  stock  of  each  type  of  precision-guided  munition. 
A  war  scenario  is  a  set  of  targets.  The  planned  portfolio  is  not  always  what  we  have  ready  when  a  war 
occurs.  It  is  affected  by  the  budget,  production  capacity  and  amount  of  time  we  have  available  to 
produce  before  the  war.  We  therefore  use  two  models.  The  first  model  calculates  an  optimal 
portfolio  given  the  constraints  of  budget  and  production  lines  available,  and  the  uncertainties  of 
preparation  time  and  war  scenario.  This  model  shows  us  the  correct  choice,  supposing  we  have 
complete  information.  The  second  model  uses  a  planned  portfolio  as  an  input,  and  calculates  its 
performance  given  values  for  budget,  production,  preparation  time,  and  war  scenario.  In  this  model, 
the  planned  portfolio  is  affected  by  available  funds,  production  and  warning  time  to  create  a  ‘war 
ready  stock’  of  munitions,  which  is  then  distributed  across  targets.  This  model  shows  us  how  the 
performance  of  a  planned  portfolio  is  be  affected  by  changes  in  the  constraints  and  uncertainties.  We 
use  one  model  to  calculate  the  optimal  portfolio  for  a  set  of  circumstances  and  use  the  other  to  see 
how  it  performs  when  those  circumstances  change.  In  addition,  we  can  use  the  second  model  to  test 
the  performance  of  a  large  number  of  portfolios  across  a  wide  variety  of  circumstances.  Both  models 
use  a  linear  program  to  simulate  Combatant  Commanders’  wartime  preferences  as  they  match 
munitions  from  the  portfolio  to  targets  in  the  scenario. 

Terminology:  Portfolios  and  Scenarios 

It  is  helpful  to  define  two  terms  that  will  be  used  throughout  the  course  of  the  illustrative  analysis. 
Portfolio  refers  to  the  munitions  portfolio,  a  reserve  stock  of  five  different  types  of  munitions.  The 
portfolio  is  the  only  policy  lever  used  in  the  approach  presented  here.  Other  policy  levers  that  affect 
the  flow  of  munitions  in  addition  to  the  stock  were  considered,  but  were  deemed  to  make  the  model 
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too  complex  for  the  purposes  of  this  dissertation.64  The  munitions  used  to  create  portfolios  are  the 
Joint  Direct  Attack  Munitions  (JDAM),  Laser  Guided  bomb  (LGB),  Tomahawk  Land  Attack 
Munition  (TLAM),  Joint  Standoff  Weapon  (JSOW),  and  the  Maverick  anti-tank  missile.  The 
characteristics  of  these  weapons  are  described  in  Appendix  1 .  A  portfolio  consists  of  some  fixed 
number  of  each  weapon,  chosen  by  the  policymaker.  This  is  by  no  means  an  exhaustive  list  of  PGMs 
—  other  munitions  were  considered.  However,  it  was  decided  to  keep  the  number  to  a  minimum  in 
order  to  ease  the  computational  requirements  of  the  model.  These  five  weapons  are  currently  the 
most  commonly  used  munitions,  and  provide  a  good  variety  in  terms  of  cost  and  effectiveness  against 
each  target  type. 

Scenario  refers  to  a  war  scenario,  which  is  defined  by  the  number  of  each  type  of  target  presented  to  a 
combatant  commander.  This  demonstrative  analysis  reduces  the  world  of  available  targets  to  four 
types.  Mobile  Hard  Targets  are  tanks.  Mobile  Soft  Targets  are  trucks,  armored  personnel  carriers 
(APCs),  jeeps  and  the  like.  Fixed  hard  targets  are  bunkers  and  hardened  facilities.  Fixed  soft  targets 
are  buildings,  radar  installation,  and  airfields.  We  recognize  that  this  is  a  very  simplified  way  of 
describing  a  war  scenario  —  there  are  many  other  factors  that  have  a  large  impact  on  the  characteristics 
of  a  war.  For  example,  terrain,  weather,  technology  and  tactics  can  each  have  a  tremendous  affect  on 
the  nature  of  a  war.  By  choosing  to  use  a  simple  definition,  we  have  sacrificed  some  resolution  so 
that  we  may  run  many  models  and  understand  how  the  constraints  and  uncertainties  affect  the 
problem  in  a  relatively  short  amount  of  time.  We  will  be  able  to  make  broad  observations,  and  this 
does  not  take  away  from  the  value  of  more  detailed  analysis. 


Steps  of  Exploratory  Analysis  and  Robust  Decision  Analysis 

The  first  practical  steps  of  an  exploratory  analysis  are  to  describe  the  scope  of  the  analysis  and 
construct  a  scenario  generator.  Defining  the  scope  includes  defining  the  questions  to  be  addressed, 
the  constraints,  uncertainties,  policy  levers,  performance  measures  and  relationships  found  in  the 
problem.  In  order  to  construct  the  scenario  generator,  one  must  then  create  a  conceptual  model; 
prioritize  the  constraints,  uncertainties,  policy  levers,  and  performance  measures;  and  finally,  code  the 
model  in  a  computer  language  appropriate  to  the  software  that  will  help  analyze  the  results.  After 
this  is  done,  we  examine  the  effects  of  changing  constraints  and  uncertainties  in  the  case  of  a  single 
war.  Then  we  generate  a  range  of  plausible  future  wars,  and  examine  the  performance  of  various 
munitions  portfolios,  looking  for  portfolios  that  perform  well  across  the  entire  range  of  wars. 

Scope  of  the  Illustrative  Analysis 

We  ask  two  questions.  First,  “how  does  changing  the  budget,  production  constraint,  and  amount  of 
preparation  time  affect  the  optimal  portfolio  given  that  we  know  which  war  will  occur?”  Next  we 


6^  Examples  of  these  levers  include  options  to  purchase  weapons  with  a  strike  price  and  date,  and  changes  in  production  processes 
like  decreased  lead  times  and  line  sharing.  Such  policies  merit  a  thorough  investigation,  but  to  do  so  in  this  paper  would  dilute  its  ability  to 
concisely  present  the  usefulness  of  the  exploratory  modeling  and  robust  analysis  methodologies. 
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ask,  “given  that  we  do  not  know  which  war  will  occur,  how  do  we  find  a  solution  that  works  well 
across  a  variety  of  possible  wars?”  In  Chapter  Two,  we  identified  several  ways  the  DoD  could 
improve  its  Munitions  Requirements  process.  We  claimed  that  by  simultaneously  considering  cost, 
production  constraints,  and  the  uncertainties  of  war,  the  DoD  can  come  to  a  better  understanding  of 
the  problem  and  possible  tradeoffs,  and  can  begin  to  make  plans  that  prepare  for  the  uncertainty  it 
faces.  The  first  part  of  illustrative  analysis  in  the  following  chapter  will  use  exploratory  modeling  and 
analysis  to  show  how  changing  the  budget  and  production  line  constraints  and  the  uncertainty  of 
how  much  time  we  have  to  prepare  for  a  war  affect  the  optimal  portfolio  in  the  case  of  a  single  war. 
But  we  also  face  the  larger  uncertainty  of  which  war  might  occur.  The  second  part  of  the  illustrative 
analysis  will  show  how  the  search  for  a  robust  solution  can  lead  us  to  a  portfolio  that  performs  well 
across  a  range  of  possible  wars  under  many  different  budgets,  production  line  setups,  and  preparation 
times. 

Constraints  and  Uncertainties 

The  major  constraints  considered  are  those  of  budget  and  production.  How  much  money  is  available 
to  spend  on  weapons?  How  many  production  lines  are  available  to  produce  each  weapon?  The 
amount  of  money  available  to  spend  and  number  of  lines  available  to  produce  munitions  could  easily 
be  considered  policy  levers  rather  than  constraints.  We  take  the  perspective  of  an  officer  in  the  Joint 
Staff  responsible  for  determining  the  munitions  requirement.  To  such  an  officer,  these  variables  are 
constraints  within  which  he  must  operate  as  he  determines  the  number  of  munitions  we  should 
order.  However,  the  results  of  this  kind  of  analysis  may  be  used  to  show  others  (e.g.  Congress),  for 
whom  budget  and  production  decisions  are  policy  levers,  how  their  decisions  affect  the  DoD’s  ability 
to  prepare  for  war. 

The  major  uncertainties  considered  here  have  to  do  with  the  wars  we  prepare  for.  Where  will  the 
wars  be  fought?  How  many  targets  will  be  there?  What  types  of  targets  must  we  hit?  How  many 
years  do  we  have  to  prepare  for  each  war?  The  answers  to  these  questions  drastically  affect  the 
definition  of  the  best  portfolio,  and  the  performance  of  sub-optimal  portfolios.  Each  war  scenario  is 
defined  by  its  size  (number  of  targets),  mix  of  targets  (proportion  of  targets  by  types),  and  number  of 
years  before  it  occurs  (preparation  time).  Each  of  these  variables  is  uncertain. 

Policy  Lever 

The  policy  lever  used  by  this  illustrative  analysis  is  a  basic  inventory  decision.  How  many  of  each 
type  of  weapon  do  we  plan  keep  in  stock?  The  inventory  levers  used  will  be  the  number  of  each  type 
of  weapon  carried  in  the  stockpile.  This  is  the  policy  decision  made  by  the  Joint  Staff  and 
recommended  to  the  President  and  then  Congress  in  the  final  budget.  The  best  answer  depends  on 
the  actual  budget,  the  war  that  occurs,  the  preparation  time  before  a  war,  and  the  number  of 
production  lines  available.  The  resulting  portfolio  ultimately  affects  wartime  performance  by 
constraining  usage  decisions  made  by  the  Combatant  Commanders  on  the  ground. 
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Performance  Measures 

We  will  use  two  simple,  aggregate  performance  measures.  We  acknowledge  that  there  are  a  large 
number  of  performance  measures,  and  choose  measures  that  may  be  made  more  specific  to  the  needs 
of  the  Joint  Staff  with  additional  data.  We  are  interested  in  the  performance  of  each  portfolio  relative 
to  the  others  across  one  or  many  war  scenarios.  We  look  first  at  the  proportion  of  targets  that  are 
covered  by  a  portfolio.  Could  the  munitions  portfolio  chosen  cover  all  targets  in  a  war?  If  not,  the 
portfolio  is  considered  unviable.  Second,  given  that  a  portfolio  can  cover  all  targets,  we  look  at  the 
number  of  munitions  it  uses  to  do  so.  A  lower  number  is  better.  We  assume  that  that  by  using  fewer 
weapons  per  target,  collateral  damage  is  kept  down,  sorties  are  reduced,  and  risks  to  pilots  and  aircraft 
are  lessened.  Given  more  data  and  time,  we  could  actually  calculate  the  performance  measures  for 
which  aggregate  weapon  usage  is  a  proxy. 

The  Variables 

Table  1  lists  the  variables  discussed  above.  There  is  a  type  of  variable  listed  here  that  has  not  yet  been 
discussed,  called  ‘Weapon  Characteristics.’  These  variables  describe  the  fundamental  differences 
between  each  weapon.  Unit  cost  is  the  average  unit  cost  of  a  munition,  as  reported  by  the  Selected 
Acquisition  Report  for  that  weapon.  The  total  unit  cost  of  the  munitions  in  a  portfolio  is  the  cost 
that  is  weighed  against  the  budget  constraint.  Production  rate  by  weapon  is  the  monthly  production 
rate  for  a  single  line  of  each  weapon.  This  number  determines  the  maximum  available  production  for 
each  weapon  over  the  course  of  the  preparation  time  available.  ‘Effectiveness  by  target  type’  is  used 
by  the  combat  simulator  to  distribute  weapons  to  targets  according  to  the  preferences  of  the 
Combatant  Commander.  The  variable  represents  the  number  of  munitions  required  to  have  a  95% 
chance  of  killing  a  target. 


Table  1 :  Variables  Used  in  Illustrative  Analysis 


Uncertain  Variables 

War  Scenario  =  Targets  (MH,  MS  FH,  FS) 
Preparation  Time  (Years  Before  the  War) 

Policy  Lever 

Planned  Portfolio  by  Munitions  Type 

Constraints 

Budget 

Number  of  Lines  per  Munitions  Type 
Weapon  Characteristics 
Unit  Costs 

Production  Rate  by  Weapon 
Effectiveness  by  target  type 


Outputs 

Percent  of  Targets  Flit 

Percent  of  Targets  Hit  with  Preferred  Munitions 
Unit  Cost  of  Munitions  Portfolio 
Weapons  Used 
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Two  Different  Models 

We  will  employ  two  different  models  —  one  calculates  the  optimal  portfolio  and  the  other  tests 
planned  portfolios.  The  first  model  calculates  an  optimal  portfolio  given  the  constraints  of  budget 
and  production  lines  available,  and  the  uncertainties  of  preparation  time  and  war  scenario.  Holding 
budget,  production,  preparation  time,  and  war  scenario  constant,  this  model  selects  the  optimal 
number  of  each  type  of  weapon  to  hold  in  the  portfolio.  It  answers  the  question,  “given  that  x  will 
occur  and  we  face  y  constraints,  what  should  we  do?”  By  varying  the  parameters  of  the  budget, 
production  lines,  preparation  time,  and  scenario,  we  may  come  to  understand  the  extent  to  which 
they  affect  the  optimal  solution  and  the  nature  of  such  effects.  This  is  in  fact  exactly  what  we  will  do 
in  the  first  part  of  the  next  chapter. 

The  second  model  is  used  to  test  the  performance  of  a  planned  portfolio.  This  model  might  be 
thought  of  as  the  inverse  of  the  one  described  above.  It  uses  a  portfolio  as  an  input,  and  calculates  its 
performance  given  values  for  budget,  production,  preparation  time,  and  war  scenario.  It  answers  the 
question,  “given  that  we  have  chosen  to  commit  to  portfolio  x,  how  does  it  perform  when  we  face 
constraints  y  and  uncertainties  z?”  where  the  constraints  are  budget  and  production  capacity  and  the 
uncertainties  are  the  preparation  time  and  war  scenario.  This  model  will  be  used  throughout  the 
course  of  the  illustrative  analysis  to  show  how  the  optimal  portfolios  perform  when  something  else 
occurs,  and  to  test  a  large  number  of  potential  portfolios  across  a  wide  range  of  scenarios  in  the  search 
for  a  robust  solution. 

The  Combat  Simulation  Module 

Both  models  use  a  combat  simulation  module,  and  we  will  describe  it  here  before  talking  about  the 
individual  models.  A  linear  program  will  be  used  for  the  combat  module.  The  purpose  of  the 
combat  model  is  to  allocate  resources  (weapons)  amongst  targets  so  as  to  maximize  (or  minimize)  the 
objective  of  combatant  commander.  In  the  first  model,  this  module  simulates  how  a  combatant 
commander  (or  his/her  surrogate  in  the  requirements  office)  would  choose  a  not  yet  determined 
portfolio  were  he/she  to  have  complete  foreknowledge  of  the  circumstances  under  which  the  portfolio 
would  be  used  (which  war,  how  much  warning,  budgetary  and  production  constraints).  In  the 
second  model,  this  module  is  used  to  simulate  how  a  combatant  commander  would  use  a 
predetermined  portfolio  under  various  circumstances.  This  use  of  the  module  is  meant  to  reflect  the 
fact  that,  although  Pentagon  policymakers  determine  the  weapons  available  for  use,  commanders  in 
the  field  decide  which  ones  to  actually  use. 

There  are  many  ways  to  optimize,  but  linear  programming  is  chosen  here  for  several  reasons.  First, 
methods  of  computation  such  as  the  use  of  calculus  and  Lagrange  multipliers  become  exceedingly 
difficult  and  memory  intensive  as  the  number  of  variables  increases.  Second,  such  methods  are  not 
practical  when  constraints  come  in  the  form  of  inequalities.  Third,  linear  programming  is  one  of  the 
most  highly  developed  and  widely  used  types  of  operations  research.  This  means  that  computations 
may  be  done  fairly  quickly,  and  that  many  tools  are  available  to  enhance  the  model’s  flexibility.  By 
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using  a  linear  program,  we  will  have  a  model  that  can  be  modified  relatively  easily  as  the  illustrative 
analysis  progresses. 

There  are  several  assumptions  that  must  be  made  in  order  to  solve  this  problem  as  a  linear  program. 

It  is  important  to  be  aware  of  them.  The  first  assumption  is  proportionality.  This  means  that  if  it 
takes  two  of  a  weapon  to  destroy  one  target  95%  of  the  time,  it  will  take  four  to  destroy  two  targets 
with  the  same  accuracy.  The  second  assumption  is  that  of  infinite  divisibility  —  that  is  that  the  input 
and  output  variables  may  be  divided  into  fractions.  Although  it  is  odd  to  think  of  using  l-_  weapons 
on  a  target,  it  is  unlikely  to  change  the  insights  gained  due  to  the  large  numbers  of  targets  and 
weapons  considered. 

The  combat  model  allocates  weapons  to  targets  in  accordance  with  the  combatant  commander’s 
objective  function.  It  is  important  to  note  that  this  model  is  an  optimizer,  and  therefore  assumes  that 
combatant  commanders  act  optimally  given  their  preferences,  resources,  and  constraints.  The 
objective  function  must  be  expressed  in  the  form,  “max/min  variable  x  subject  to  the  constraints  y.” 
The  constraints  may  be  equality  constraints  or  inequalities.  For  example,  suppose  that  we  believe 
that  the  commanders  want  to  (1)  destroy  all  the  targets  with  95%  accuracy  and  (2)  hit  them  with  the 
best  available  weapons.  The  objective  would  be  phrased  as,  “minimize  the  number  of  weapons  used 
subject  to  the  constraints  that  all  targets  are  destroyed  with  95%  accuracy  and  that  no  more  weapons 
are  used  than  are  in  stock.”  With  this  objective,  the  model  would  be  solved  so  that  commanders  first 
try  to  hit  all  the  targets.  If  the  budget,  production  capacity  (or  in  the  case  of  the  second  model, 
portfolio)  is  too  low  to  do  so,  they  will  hit  as  many  as  possible.  If  all  targets  can  be  hit,  commanders 
will  try  to  hit  them  using  the  fewest  number  of  weapons  —  this  is  equivalent  to  using  the  most 
accurate  weapon  for  each  target. 

One  of  the  benefits  of  using  a  linear  program  is  the  ease  with  which  the  constraints  and  objectives 
may  be  changed.  This  makes  it  possible  to  modify  the  model  after  we  have  learned  more  about  the 
problem,  or  have  different  questions  to  ask.  It  would  be  easy  to  add  a  constraint  specifying  that  some 
stock  must  be  held  in  reserve  in  case  of  another  war  —  or  that  a  certain  number  of  SAM  sites  (fixed 
soft  targets)  must  be  destroyed  in  order  to  complete  the  mission.  Additionally,  the  commander’s 
objective  can  be  changed.  For  instance,  if  the  commanders  are  more  interested  in  saving  pilot’s  lives 
than  in  the  effectiveness  of  the  weapon  per  se,  we  could  make  the  objective  that  of  minimizing  loss  of 
life.  All  that  would  be  needed  is  an  estimate  of  the  number  of  sorties  flown  (derived  from  the 
number  of  each  type  of  weapon  used)  and  the  probability  of  downed  aircraft  per  sortie. 

The  First  Model:  Used  to  Find  an  Optimal  Portfolio 

The  figure  below  provides  a  concise  representation  of  the  important  relationships  in  the  optimizing 
model.  This  model  uses  the  combat  module  to  calculate  the  optimal  munitions  portfolio,  given  the 
inputs  of  maximum  budget,  preparation  time,  production  lines,  and  war  scenario. 
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Figure  1 :  Model  Used  to  Find  Optimal  Portfolios 
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This  optimization  assumes  that  the  commander’s  objective  is  (1)  hit  all  targets  and  (2)  use  the  best 
possible  (lowest  sortie)  weapon  on  each  target  whenever  possible.  As  we  discussed  above,  low  weapon 
usage  is  a  proxy  for  other  good  things,  like  risk  of  collateral  damage  and  pilot  loss. 


The  Second  Model:  Used  to  Test  Portfolios  Across  Scenarios  and  Constraints 

The  figure  below  provides  a  concise  representation  of  the  important  relationships  in  the  testing 
model.  This  model  uses  a  planned  portfolio  as  an  input.  The  planned  portfolio  is  then  transformed 
into  a  war  ready  portfolio  as  it  is  affected  by  the  budget/production  constraints  and  the  uncertainty  of 
preparation  time.  This  model  uses  the  combat  module  to  simulate  how  a  combatant  commander 
would  use  the  war  ready  portfolio  for  a  given  war  scenario. 


Figure  2:  Model  Used  to  Test  Planned  Portfolios 
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The  two  major  inputs  are  a  planned  portfolio  —  designated  as  a  number  of  each  type  of  weapon  that 
we  would  like  to  have  in  stock  —  and  a  war  scenario  —  the  target  mix  for  a  specific  war.  The  planned 
portfolio  operates  through  three  major  variables  —  the  budget,  number  of  production  lines  and 
warning  time  -  in  order  to  calculate  the  number  of  weapons  available  at  the  time  of  a  conflict.  The 
budget  affects  the  number  of  weapons  that  are  actually  bought  —  as  opposed  to  the  number  planned. 
If  the  budget  is  sufficient  to  pay  for  the  planned  inventory,  then  the  entire  planned  inventory  is 
available.  However,  if  the  budget  is  less  than  the  cost  of  the  planned  inventory,  the  inventory  is  cut 
according  to  an  algorithm.  The  default  algorithm  cuts  planned  inventory  evenly  across  all  munitions 
in  order  to  meet  the  budget  constraint.  Other  algorithms  could,  for  example,  cut  expensive  weapons 
first.  The  amount  of  warning  time,  in  conjunction  with  the  available  production  lines,  determines 
the  amount  of  inventory  available  from  the  assembly  lines.  If  the  plan  calls  for  more  inventory  of  a 
weapon  than  can  be  produced  before  the  time  of  the  war,  the  war  ready  stock  is  cut  back  to  the 
maximum  number  available. 

The  Combat  Module  (a  linear  programming  model)  determines  how  a  combatant  commander  would 
use  the  combat  ready  portfolio  against  the  input  scenario.  In  this  case,  the  module  calculates  a 
portfolio  that  minimizes  the  number  of  munitions  used  by  matching  munitions  to  targets  found  in 
the  scenario  subject  to  the  following  constraints: 

1.  All  targets  are  covered  (hit  with  enough  weapons  to  guarantee  a  95%  chance  of  destruction) 

2.  The  combatant  commander  does  not  use  more  of  any  weapon  than  the  war  ready  stock 
allows. 

The  module  provides  us  with  two  performance  outputs:  the  proportion  of  targets  covered  (destroyed 
with  a  95%  probability)  and  the  number  of  munitions  used. 


CHAPTER  SEVEN 

Illustrative  Analysis  of  A  Capabilities -based  Approach 


Overview 

Here  we  perform  an  illustrative  analysis  using  the  approach  described  in  the  last  chapter,  which 
allows  planners  to  find  solutions  that  provide  a  wide  range  of  capabilities  within  an  economic 
framework  under  conditions  of  uncertainty.  The  first  part  of  this  chapter  uses  exploratory  analysis, 
an  extreme  form  of  traditional  sensitivity  analysis,  to  show  that  the  constraints  of  cost,  production 
and  preparation  time  dramatically  affect  the  optimal  portfolio  of  weapons  for  a  single  scenario.  But 
we  never  know  which  scenario  will  occur  -  and  different  portfolios  (capabilities)  work  better  for  some 
scenarios  than  others.  The  second  part  of  this  chapter  demonstrates  that  the  use  of  robust  decision¬ 
making  techniques  makes  it  possible  to  analyze  the  effects  of  these  constraints  across  multiple 
scenarios  and  create  improved  and  flexible  portfolios. 


Part  One:  Exploratory  Analysis 

The  Budget  Constraint 

In  the  MR  process,  a  surrogate  combatant  commander  (the  requirements  office  of  each  service) 
determines  the  combat  requirement  with  the  goal  of  winning  the  war  in  the  best  possible  way.  The 
munitions  budget  is  not  an  integral  part  of  the  office’s  calculations.  If  a  combatant  commander  were 
told  in  advance  how  much  money  he/she  had  to  spend,  he/she  would  prefer  different  mixes  of 
weapons  at  different  levels  of  funding.  A  commander  given  one  billion  dollars  would  buy  a  different 
mix  of  weapons  than  a  commander  given  ten  billion  dollars.  Under  the  current  MR  process,  it  is 
difficult,  if  not  impossible  to  see  these  preferences. 

Figure  3  below  shows  the  preferred  weapon  mixes  for  a  large,  hypothetical  South  East  Asia  war 
scenario  over  forty  different  budget  scenarios  ranging  from  $100  million  to  $20  billion. 
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Figure  3:  Combatant  Commanders’  Preferred  Portfolios  By  Budget  Level,  SE  Asia  Scenario 
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The  goal  of  the  commander  is  to  minimize  the  number  of  weapons  used,  subject  to  the  constraint 
that  all  targets  are  hit.  For  every  budget  level,  the  commander’s  first  priority  is  to  cover  all  the  targets. 
His/her  second  priority  is  to  use  as  few  weapons  as  possible.  Weapon  usage  is  treated  here  as  a  proxy 
for  other  performance  measures.  The  assumption  is  that  by  using  fewer  weapons  per  target,  collateral 
damage  is  kept  down65,  sorties  are  reduced,  and  risks  to  pilots  and  aircraft  are  lessened. 

At  high  levels  of  funding,  the  commander  prefers  to  use  Mavericks,  TLAMs  and  JSOWs.  As  funding 
decreases,  the  commander  replaces  these  weapons  with  less  expensive  LGBs  and  JDAMs.  The 
tradeoff  is  that  it  takes  more  of  the  cheaper  weapons  to  ensure  equivalent  coverage  of  the  targets. 

Figure  4  shows  the  tradeoffs  in  a  more  direct  way,  displaying  the  performance  of  the  cost  constrained 
portfolios  in  terms  of  the  percentage  of  targets  hit  and  the  percentage  of  targets  that  are  hit  with  the 
‘best’  weapon.66  By  changing  the  composition  of  the  portfolio,  a  combatant  commander  is  able  to 
hit  all  the  targets  with  as  little  as  $2.1  billion.  However,  such  a  solutions  uses  approximately  twice 


65  Collateral  damage  calculation  is  very  complex  issue  and  this  is  an  admittedly  very  simplified  assumption.  There  are  many  ways  to 
lessen  collateral  damage  that  have  little  to  do  with  which  weapon  is  used  -  aiming  at  specific  parts  of  a  target  or  using  different  fuses,  for 
example. 

66  The  ‘best’  weapon  is  defined  as  the  weapon  with  the  highest  single  shot  probability  of  destroying  a  target.  Using  the  best  weapon 
is  preferred  as  it  destroys  the  target  faster  and  with  less  follow  on  strikes,  increasing  the  speed  of  an  operation  and  reducing  pilot  risk  and 
collateral  damage. 
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the  number  of  weapons  (80,000  vs.  40,000)  as  the  $15  billion  solution  and  hits  none  of  the  targets 
with  the  best  weapon. 

Figure  4:  Performance  of  Preferred  Portfolios  in  Terms  of  Targets  Hit  and  Targets  Hit  With  the 

‘Best’  Weapons,  SE  Asia  Scenario 


Budget  ($B) 

Percent  of  Targets  Covered  ™  “  Percent  of  Targets  Covered  by  Highest  pK  Weapon 


Under  process  we  use  today,  a  surrogate  combatant  commander  (the  requirements  office)  determines 
the  combat  requirement  with  the  goal  of  winning  the  war  in  the  best  possible  way.  The  result  is 
equivalent  an  ‘un-cost-constrained’  optimal  -  the  far  right  portfolios  in 

Figure  3,  which  cost  approximately  $15  billion.  Entirely  separate  groups  of  officers  are  then  tasked  to 
fit  this  requirement  to  the  budget.  Unless  ad  hoc  changes  are  made  to  the  requirement,  the  results  of 
such  planning  can  be  disastrous.  As  an  example,  the  following  figures  show  what  happens  when 
planners  decide  to  use  an  unconstrained  solution,  but  are  forced  to  cut  it  due  to  budget  constraints. 

In  this  example,  if  there  is  not  enough  money  available  to  pay  for  the  unconstrained  portfolio,  the 
order  is  cut  uniformly  across  weapons  to  match  the  available  budget. 

Figure  5  represents  this  visually.  The  solutions  at  the  far  right  are  the  ones  that  the  combatant 
commander  would  choose,  were  he/she  given  no  budget  constraint.  As  the  budget  declines,  the  order 
is  cut  back  equally  across  all  weapons. 
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Figure  5:  Unconstrained  Portfolio  Subsequently  Modified  by  Budget,  SE  Asia  Scenario 


Actual  Budget  ($B) 


Figure  6  below  compares  the  performance  of  the  unconstrained  portfolio  with  the  cost  constrained 
portfolios  across  the  same  twenty  budget  scenarios.  It  is  readily  apparent  that  there  is  a  tremendous 
benefit  to  considering  the  available  budget  as  a  constraint  during  the  requirements  process.  By  doing 
so,  commanders  are  able  to  cover  all  the  targets  with  as  little  as  $2  billion.  On  the  other  hand,  the 
unconstrained  portfolio  performs  poorly  across  budget  scenarios.  If  the  actual  budget  is  greater  than 
or  equal  to  the  cost  of  the  unconstrained  portfolio  (about  $15  billion),  all  the  targets  are  hit,  and  hit 
with  the  best  weapon.  But  the  number  of  targets  hit  decreases  linearly  with  the  budget  because  the 
combatant  commanders  and  their  surrogate  requirements  officers  are  not  able  to  trade  away  preferred 
weapons  in  order  to  cover  all  the  targets.  The  result  is  a  portfolio  that  performs  worse  over  every 
budget  scenario  less  than  $15  billion,  and  no  better  for  scenarios  over  $15  billion. 
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Figure  6:  Comparing  Performance  of  Unconstrained  Portfolio  with  Cost  Constrained  Portfolios 

Across  Budget  Levels,  SE  Asia  Scenario 
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The  example  given  here  is  extreme,  but  not  improbable.  The  practice  of  distributing  decrements  to 
budgets  equally  across  programs,  colloquially  known  as  ‘salami  slicing,’  is  not  uncommon  within  the 
DoD.  Budgeters  do  not  understand  how  a  commander’s  preferences  might  change  with  a  smaller  or 
larger  budget.  They  do  understand  that  they  must  fit  all  DoD  programs  into  the  budget.  The  MR 
process  hedges  against  salami  slicing  by  inflating  the  requirement  so  that  each  ‘slice’  has  a  smaller 
chance  of  damaging  the  portfolio  to  a  point  where  performance  is  unacceptable.  Requirements 
officers  are  excellent  war  planners  and  have  an  intimate  knowledge  of  weapon  effectiveness.  If  they 
were  given  some  idea  of  the  true  budget  before  creating  the  initial  requirement,  the  requirement  and 
budget  would  match  up  —  and  budgeters  would  not  need  to  slice.  Furthermore,  the  DoD  would  be 
getting  the  best  possible  portfolios  for  its  money,  and  its  requirements  would  have  some  credibility. 

The  Production  Constraint 

A  second  constraint  that  is  not  considered  concurrently  with  either  the  requirements  generation 
process  or  buying  decisions  is  the  production  constraint.  The  example  provided  above  assumes  that 
we  can  produce  as  many  of  each  weapon  type  as  are  needed.  The  reality  is  that  we  only  have  so  many 
production  lines  for  each  weapon,  and  that  those  lines  are  capable  of  providing  only  a  fixed  amount 
of  production  over  the  course  of  a  year.  For  the  purposes  of  the  following  charts,  we  assume  a  7-year 
planning  cycle,  meaning  that  each  line  can  produce  6  years’  worth  of  weapons  before  the  conflict. 
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Figure  7  below  shows  how  the  optimal  portfolios  change  depending  on  whether  we  have  1,  2  or 
unlimited  production  lines  available  for  each  weapon. 

Figure  7:  Optimal  Portfolios  by  Available  Production  Lines,  SE  Asia  Scenario 


One  Line  Per  Weapon 


0.1  1.59  3.09  4.58  6.07  17.56  9.06  10.5  12  13.5  15  16.5  18  19.5 

Budget  ($B) 

□  MavsUsed  □TLAMslIsed  DJSOWsUsed  DLGBsUsed  BJDAMsUsed 


Two  Lines  Per  Weapon 


ip- 


1.59  3.09  4.58  6.07  7.56  9.06 


10.5  12  13.5  15  16.5  18  19.5 


Budget  ($B) 


□  MavsUsed  ■TLAMsUsed  QJSOWsUsed  DLGBsUsed  BJDAMsUsed 


Illustrative  Analysis  of  A  Capabilities-based  Approach  45 


Unlimited  Production 


90000 
80000 
«  70000 

.•§  60000 

C 

i  50000 
°  40000 
f  30000 
z  20000 
10000 
0 


0.1  1.6  3.1  4.6  6.1  7.6  9.1  10.5  12.0  13.5  15.0  16.5  18.0  19.5 

Budget  ($B) 


□  MavsUsed  BTLAMsUsed  DJSOWsUsed  nLGBsUsed  ■  JDAMsUsed 


The  optimal  portfolio  changes  depending  on  available  production.  When  there  are  not  many  lines 
(one  line  per  weapon),  the  commander  uses  more  JDAMs  and  LGBs  because  one  line  can  build  many 
of  these  weapons.  He  would  prefer  to  use  other  weapons,  but  cannot  buy  more  than  are  available. 
Maverick,  JSOW,  and  TLAM  buys  are  capped  by  the  number  that  can  be  produced  in  six  years. 

With  unlimited  production  (i.e.  no  production  constraint),  the  commander  would  choose  to  buy 
many  more  Mavericks  and  JSOWS  at  low  budget  levels  and  increase  TLAM  buys  at  high  budget 
levels. 

Limited  production  decreases  the  effectiveness  of  each  dollar  spent  on  weapons  and  places  an  upper 
bound  on  the  amount  of  money  that  should  be  spent  on  munitions.  Figure  8  shows  how  the  number 
of  munitions  used  changes  depending  on  the  budget  and  production  constraint.  The  figure  shows 
these  data  for  budget  levels  that  allow  complete  target  coverage  (budgets  greater  that  $2  billion). 
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Figure  8:  Performance  in  Terms  of  Munitions  Used  by  Budget  Level  for  3  Different  Production 

Constraints,  SE  Asia  Scenario 


- Unlimited  - 2  Lines  Per  Weapon  - 1  Line  per  Weapon 

As  the  production  constraint  is  loosened,  combatant  commanders  are  allowed  to  choose  portfolios 
that  can  hit  all  the  targets  with  fewer  weapons.  If  there  is  only  one  production  line  for  each  weapon, 
a  commander  with  $7  billion  could  hit  all  the  targets  with  about  57,000  munitions.  However,  if 
there  are  two  production  lines  available,  he  could  do  it  with  about  50,000  munitions  —  without 
changing  his  budget. 

Furthermore,  we  see  that  with  one  production  line  for  each  weapon,  there  is  no  point  in  spending 
more  than  $7  billion.  Spending  more  does  not  improve  performance  because  the  trades  the 
commander  would  like  to  make  are  limited  by  production!  With  two  production  lines,  there  is  no 
reason  to  spend  more  than  $10  billion  and  with  unlimited  production,  there  is  no  reason  to  spend 
more  than  $15  billion. 

Preparation  Time 

A  final  constraint  is  preparation  time  —  the  number  of  years  before  the  war  occurs.  Thus  far,  we  (like 
the  DoD  requirements  office)  have  assumed  that  the  war  occurs  7  years  in  the  future.  In  reality,  we 
do  not  know  when  a  war  will  occur.  It  could  happen  in  7  years,  or  it  could  happen  tomorrow.  The 
preparation  time  constraint  operates  in  a  similar  manner  to  the  production  constraint.  It  limits  the 
number  of  munitions  available  at  the  time  of  the  conflict. 


Illustrative  Analysis  of  A  Capabilities-based  Approach  47 


If  we  create  a  portfolio  believing  that  we  have  6  years  of  production  available  and  the  war  sooner  we 
are  forced  to  go  to  war  with  a  partial  portfolio.  The  result  is  poor  performance.  Table  2  shows  the 
maximum  available  production  over  six  years  if  we  have  one  production  line  for  each  weapon. 


Table  2:  Maximum  Production:  One  Line  per  Weapon,  Six  Years  of  Preparation 


JDAM 

TLAM 

JSOW 

LGB 

Maverick 

216000 

2880 

7200 

115200 

10800 

If  cost  were  no  object,  a  commander  would  choose  the  following  portfolio  at  a  price  of  $7.2  billion: 


Table  3:  Optimal  Portfolio:  One  Line  Per  Weapon,  Six  Years  of  Preparation 


JDAMs 

TLAMs 

ISOW 

LGB 

Maverick 

11250 

2880 

7200 

31860 

10800 

Note  that  TLAM,  Maverick  and  JSOW  inventories  are  each  constrained  by  available  production  as 
discussed  above.  This  portfolio  would  cover  all  targets  and  use  63,990  weapons  in  the  process.  Now, 
suppose  the  war  occurs  two  years  earlier  than  expected.  Maximum  available  production  drops  by  1/3 
to: 


Table  4:  Maximum  Production:  One  Line  per  Weapon,  Four  Years  of  Preparation 


JDAM 

TLAM 

JSOW 

LGB 

Maverick 

144000 

1920 

4800 

76800 

7200 

The  portfolio  we  originally  chose  will  now  be  missing  some  weapons  —  namely  the  TLAMs,  JSOWs, 
and  Mavericks  that  have  yet  to  be  produced.  Table  5  compares  the  originally  planned  portfolio  with 
the  munitions  that  are  actually  available  for  combat. 
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Table  5:  Comparing  the  Six-Year  Planned  Portfolio  With  Combat  Ready  Munitions  After  Four  Years 


JDAMs 

TLAMs 

JSOW 

LGB 

Maverick 

Original  Planned  Portfolio 

11250 

2880 

7200 

31860 

10800 

Actual  Munitions  Ready  for  Combat 

11250 

1920 

4800 

31860 

7200 

The  result  is  a  portfolio  that  only  hits  77%  of  the  targets  (shown  in  Table  7).  If  the  combatant 
commander  knew  that  there  would  only  be  four  years  to  prepare  for  the  war,  he/she  would  have 
chosen  the  following  solution  at  a  cost  of  $5.96  billion: 


Table  6:  Optimal  Portfolio:  One  Line  Per  Weapon,  Four  Years  of  Preparation  Time 


JDAMs 

TLAMs 

JSOW 

LGB 

Maverick 

11250 

1920 

4800 

43740 

7200 

If  the  commander  knew  in  advance  that  he/she  would  have  only  4  years  of  preparation  time,  the 
JSOWs,  Mavericks  and  TLAMs  would  be  replaced  by  increased  orders  of  LGBs.  Figure  9  shows  this 
visually. 
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Figure  9:  Visual  Representation  of  Portfolios:  Six  Year  Plan  after  Four  and  Six  Years,  Four  Year  Plan 


□  Plan  for  Six  Years  of  Preparation  Time  ■  Six  Year  Plan  After  Four  Years 
■  Plan  for  Four  Years  of  Preparation  Time 

The  four-year  plan  portfolio  covers  all  targets,  using  68,908  weapons.  The  difference  in  performance 
between  the  six  year  and  four  year  plans  is  shown  in  Table  7  below. 


Table  7:  Performance  of  Six  Year  and  Four-Year  Planned  Portfolios  After  Four  Years 


Percent  of  Targets  Covered 

Munitions  Used 

Six  Year  Plan  After  Four  Years 

77% 

63,990 

Plan  for  Four  Years  of  Preparation  Time 

100% 

68,909 

What  does  this  tell  us?  First,  that  it  can  be  dangerous  to  assume  the  war  will  occur  at  any  set  point  in 
the  future  and  plan  only  for  that  occurrence.  Second,  there  are  very  specific  strategies  that  can 
improve  performance  in  the  event  of  an  early  war.  By  exploring  the  optimal  solutions,  we  found  that 
if  we  knew  an  early  war  was  coming,  we  would  buy  more  LGBs.  The  implication  here  is  that  if  we 
are  concerned  about  an  early  war,  we  can  selectively  overstock  the  weapons  that  will  be  useful.  This 
is  a  far  cry  from  an  ad  hoc  inflation  of  the  requirement.  We  have  direct  evidence  supporting  the  need 
for  more  of  a  certain  type  of  weapon  to  hedge  against  an  early  war.  Furthermore,  we  have  a  chart 
showing  the  benefits  of  doing  so. 
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Scenario  Uncertainty  Complicates  Matters 

Thus  far  we  have  shown  that  it  is  possible  to  rigorously  examine  the  effects  of  changing  the 
constraints  of  budget,  production  and  preparation  time  for  a  single  war  scenario.  In  doing  so,  we 
have  learned  several  valuable  lessons.  First,  it  is  important  to  include  the  budget  in  the  MR  process  — 
and  allow  those  with  intimate  knowledge  of  weapon  effectiveness  to  do  the  integration.  Second,  it  is 
important  to  include  production  constraints  in  one’s  evaluation,  as  the  number  of  lines  available  can 
increase  performance  at  lower  budget  levels  and  cap  it  at  higher  levels.  Third,  we  should  be  careful 
about  our  assumptions  regarding  warning  time,  as  surprises  can  severely  limit  the  performance  of  a 
portfolio.  At  the  same  time,  we  have  created  charts  that  can  show  policymakers  exactly  how  the 
optimal  solution  and  its  performances  changes  with  budget,  production,  or  preparation. 

It  is  not  difficult  to  examine  the  effects  of  varying  constraints  for  a  single  scenario.  But  planners  face 
an  even  larger  problem  —  they  must  be  prepared,  not  for  any  one  particular  war,  but  for  the  war  that 
actually  occurs.  We  do  not  know  where  the  next  war  might  occur,  and  the  penalty  for  preparing  for 
the  wrong  one  can  be  steep.  It  is  therefore  important  to  consider  the  cross  scenario  performance  of  a 
munitions  portfolio.  What  happens  if  we  prepare  for  one  scenario,  and  another  one  occurs?  We 
want  a  munitions  portfolio  that  will  serve  us  well  across  a  wide  range  of  scenarios  and  constraints. 
When  we  must  operate  within  the  constraints  of  cost,  production,  and  preparation  time,  it  is  very 
difficult  -  if  not  impossible  —  to  mathematically  construct  a  solution  that  is  optimal  across  scenarios. 

The  following  sections  will  demonstrate  that  the  robust  decision  making  methodology  allows  us  to 
easily  find  portfolios  that  perform  well  across  a  wide  range  of  scenarios  and  changing  constraints. 

The  optimal  solution  for  one  war  may  perform  very  poorly  if  another  war  occurs.  By  identifying  a 
measure  of  regret  and  seeking  solutions  that  minimize  regret  between  scenarios,  we  can  choose 
portfolios  that  do  not  perform  the  best  in  any  one  scenario,  but  that  perform  very  well  across  all  of 
them.  Regret  is  the  difference  between  the  expected  performance  of  a  strategy  being  tested  on  a 
scenario  and  the  performance  of  the  optimal  strategy  under  perfect  information. 

Characterizing  Scenario  Uncertainty 

Before  we  proceed,  we  must  settle  upon  an  appropriate  characterization  of  the  uncertainty  that  we 
face.  Here  we  show  that  using  current  ‘Axis  of  Evil’  scenarios  to  describe  scenario  uncertainty  may  be 
misleading  as  a  large  disparity  in  scenario  size  leads  to  the  myopic  conclusion  that  it  is  best  to  prepare 
for  the  largest  scenario.  Suppose  that  the  DoD  planning  guidance  tells  us  to  be  prepared  for  the 
following  scenarios  -  DPRK  (North  Korea),  Syria,  Iraq  (2001)  and  Iran  and  we  are  given  a  target  list 
by  the  Defense  Intelligence  Agency  (DIA).  Such  a  target  list  might  look  like  Table  8  below. 
(Numbers  are  taken  from  unclassified  NAVY  Periscope  and  Strategic  Studies  Institute  databases) 
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Table  8:  Scenario  Descriptions  -  Number  of  Targets  By  Type 


Mobile  Hard 

Mobile  Soft 

Fixed  Hard 

Fixed  Soft 

Total 

DPRK 

4700 

9990 

1030 

10890 

26610 

Syria 

4500 

6050 

570 

490 

11610 

Iraq  -2001 

3100 

3150 

950 

1900 

9100 

Iran 

1170 

3050 

1540 

1050 

6810 

Uncertainly  is  characterized  here  simply  as  uncertainty  about  which  scenario  will  occur.  Already,  we 
can  see  that,  with  the  exception  of  the  large  number  of  fixed  hard  targets  in  Iran,  the  targets  found  in 
Syria,  Iraq  and  Iran  are  a  subset  of  the  targets  found  in  North  Korea.  Assuming  we  face  no 
constraints,  the  optimal  solution  for  each  scenario  would  look  like  Table  9  below. 


Table  9:  Optimal  Unconstrained  Portfolio  By  Scenario 


JDAM 

TLAM 

JSOW 

LGB 

Maverick 

DPRK 

0 

11920 

19980 

0 

9400 

Syria 

0 

1060 

12100 

0 

9000 

Iraq  (2001) 

0 

2850 

6300 

0 

6200 

Iran 

0 

2590 

6100 

0 

2340 

Table  10  displays  the  performance  of  the  four  optimal  portfolios  across  each  war  scenario  in  terms  of 
the  percentage  of  targets  that  are  covered.  Table  1 1  displays  the  performance  of  the  same  portfolios 
across  the  wars  in  terms  of  the  number  of  weapons  used. 


Table  10:  Percentage  of  Targets  Covered:  Preparation  vs.  Actual  Scenario 


Portfolio 

DPRK 

Syria 

Iraq  2001 

Iran 

DPRK  Unconstrained 

100 

100 

100 

100 

Syria  Unconstrained 

44 

100 

100 

100 

Iraq  200 1  Unconstrained 

34 

63 

100 

100 

Iran  Unconstrained 

26 

46 

75 

100 
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Table  11:  Amount  of  Ordinance  Dropped,  Excluding  Scenario/Strategy  Combinations  with  Missed 

Targets 


DPRK 

Syria 

Iraq  2001 

Iran 

DPRK  Unconstrained 

41300 

22160 

15350 

11030 

Syria  Unconstrained 

- 

22160 

17140 

13040 

Iraq  200 1  Unconstrained 

- 

- 

15350 

11030 

Iran  Unconstrained 

- 

- 

- 

11030 

The  DPRK  portfolio  is  the  most  robust  —  faring  the  best  across  all  scenarios,  both  in  terms  of  number 
of  targets  hit  and  the  number  of  weapons  used.  In  fact,  it  does  as  well  as  the  Syria,  Iran  and  Iraq 
optimal  portfolios  for  both  performance  measures.  The  Iran  portfolio,  on  the  other  hand,  is  not 
robust.  It  performs  quite  well  if  the  Iran  scenario  occurs,  but  very  poorly  in  the  event  of  any  other 
scenario.  The  reason  why  the  DPRK  portfolio  is  robust  is  straightforward.  DPRK  is  by  far  the 
largest  scenario,  as  demonstrated  by  Table  8.  The  targets  of  the  other  scenarios  are  subsets  of  the 
DPRK  targets.  The  optimal  portfolios  for  the  other  scenarios  are  subsets  of  the  DPRK  optimal 
portfolio. 

This  tells  us  that  it  is  far  better  to  prepare  for  a  large  scenario  than  to  prepare  for  a  small  one.  It  is 
extremely  expensive  to  prepare  for  such  a  large  scenario,  especially  if  we  want  the  best  possible 
portfolio.  Additionally,  we  are  not  simply  interested  robustness  to  scenario  size.  We  could  go 
further  here  and  examine  how  changing  the  constraints  might  alter  the  robust  portfolio  -  but  to  do 
so  would  not  be  interesting.  The  DPRK  scenario  is  so  much  larger  that  the  others  that  it  will  always 
be  the  most  robust  —  preparing  for  anything  smaller  (e.g.  any  of  the  other  scenarios)  results  in 
disaster.  The  hidden  assumption  here  is  that  the  largest  scenario  will  look  like  DPRK.  Military 
tactics  and  force  structures  change  —  especially  now.  Preparing  for  the  largest  current  threat  may  not 
be  enough.  What  if  DPRK  ceases  to  be  our  largest  adversary?  What  if  we  must  fight  simultaneous 
wars  against  smaller  adversaries  whose  force  structure  is  entirely  different  from  that  of  DPRK?  We 
want  a  portfolio  that  is  robust  not  just  to  war  size,  but  also  to  target  type  uncertainty.  We  also  want 
to  be  able  to  make  compelling  arguments  about  why/when  we  should  choose  certain  munitions  over 
others  so  that  intelligent  decisions  can  be  made  within  a  constrained  budgetary  environment. 

To  this  end,  we  will  characterize  our  uncertainty  as  ‘target  type’  uncertainty.  We  assume  that  most  of 
the  uncertainties  associated  with  our  own  tactics  and  our  adversaries’  identity,  equipment  and  tactics 
can  be  summed  up  by  the  types  of  targets  that  need  to  be  hit.  We  want  to  find  a  cost  effective 
portfolio  that  is  robust  to  target  type  uncertainty.  In  order  to  do  so,  we  look  for  solutions  that 
perform  well  across  five  equally  sized  scenarios  made  up  of  different  target  types.  The  scenarios  used 
are  slightly  larger  in  size  than  the  DPRK  scenario  above.  Each  contains  30,000  targets  and  is  shown 
in  Table  12  below.  Four  of  the  scenarios  are  designed  to  test  targeting  extremes.  These  four  contain 
70%  of  one  target  type  (mobile  hard,  mobile  soft,  fixed  hard,  or  fixed  soft)  and  10%  of  each  of  the 
other  target  types.  The  fifth  scenario  is  a  balanced  one  that  contains  25%  of  each  target  type. 
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Table  12:  Scenarios  To  Be  Used 


Fixed  Soft 

Fixed  Hard 

Mobile  Soft 

Mobile  Hard 

Buildings 

21000 

3000 

3000 

3000 

Bunkers 

3000 

21000 

3000 

3000 

APCs 

3000 

3000 

21000 

3000 

Tanks 

3000 

3000 

3000 

21000 

Balanced 

7500 

7500 

7500 

7500 

We  have  chosen  to  characterize  the  uncertainty  we  face  as  target  type  uncertainty  and  have  picked 
scenarios  that  reflect  the  extreme  target  mixes  we  might  face  in  the  futures.  We  have  also  assumed 
that  we  will  face  a  war  or  combination  of  wars  that  will  include  no  more  that  30,000  targets 
(although  the  solutions  given  below  are  likely  scalable  to  larger  or  smaller  wars).  The  next  section 
begins  our  exposition  of  the  robust  decision  making  methodology. 


Part  Two:  Searching  for  Robust  Solutions 

We  search  for  a  robust  solution  in  cases  where  it  is  impossible  to  find  a  globally  optimal  solution 
across  scenarios.  In  this  case,  we  could  find  an  optimum  quite  easily  if  we  chose  to  place  a  probability 
distribution  on  the  event  of  each  war  occurring.  However  uncertainty  about  the  war  we  will  face  in 
two,  four  or  seven  years  cannot  be  fit  to  any  distribution.  The  distribution  would  simply  be  an 
opinion  of  what  is  most  likely  to  occur.  In  fact,  we  place  such  probabilities  on  the  occurrence  of  wars 
right  now  —  when  the  Defense  Planning  Guidance  says  to  prepare  for  a  war,  it  is  implicitly  placing  a 
100%  probability  on  the  occurrence  of  that  war.  That  probability  is  derived  from  discussions  and 
opinions  within  the  office  of  the  Secretary  of  Defense  for  Policy  and  Planning.  The  war  may  occur, 
or  it  may  not.  But  it  is  impossible  to  properly  estimate  the  probability  of  any  single  war  occurring 
sometime  in  the  future.  We  will  show  that  by  searching  for  robust  portfolios  we  can  find  solutions 
that  are  successful  across  a  range  of  wars  while  refraining  from  making  conjectures  about  the  future. 
Furthermore,  successful  robust  portfolios  can  be  created  within  the  context  of  budget,  production 
and  preparation  constraints,  and  the  tradeoffs  involved  may  be  easily  understood. 

This  illustrative  analysis  finds  robust  portfolios  by  creating  a  large  pool  of  portfolios  and  testing  them 
in  each  of  the  scenarios.  There  are  many  ways  to  create  such  a  pool.  One  way  might  be  to  run 
through  all  the  permutations  of  weapons  stocks  based  on  current  production.  However,  doing  so 
would  create  too  many  portfolios  to  test  in  a  reasonable  amount  of  time.  We  use  a  Latin  Hypercube 
generator  to  create  10,000  random  portfolios,  constrained  by  the  amount  of  production  available. 

The  number  of  each  type  of  weapon  found  in  a  portfolio  is  a  random  number  from  0  to  the 
maximum  available  production  based  on  the  production  constraints  and  warning  time  being  tested. 
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Each  portfolio  is  then  tested  in  each  scenario.  The  ‘test’  is  a  linear  program  that  allocates  a  portfolio’s 
munitions  across  targets  in  a  way  that  simulates  how  they  would  be  used  by  a  combatant  commander. 
The  first  goal  is  to  cover  all  targets,  and  the  second  goal  is  to  use  as  few  weapons  as  possible.  The 
major  performance  measures  are  whether  or  not  all  the  targets  are  covered  and,  if  they  are,  how  many 
weapons  need  to  be  dropped  to  do  so. 

The  first  thing  we  do  when  choosing  the  most  robust  portfolio  is  drop  all  portfolios  that  cannot  cover 
targets  in  every  scenario.  These  portfolios  do  not  meet  the  most  basic  criteria  for  success.  Then  we 
search  for  portfolios  that  use  the  least  number  of  weapons  across  scenarios.  Some  scenarios  are  more 
difficult  than  others  and  portfolios  do  not  perform  equally  across  scenarios.  For  that  reason  we 
compare  the  number  of  weapons  a  portfolio  uses  to  cover  all  targets  in  each  scenario  with  that 
scenario’s  optimal  portfolio  weapon  usage  and  calculate  the  difference.  That  difference  is  called  the 
regret  -  it  answers  the  question,  “how  much  better  could  I  have  done  in  a  perfect  world?”  Then,  we 
rank  the  portfolios  by  the  maximum  regret  across  scenarios.  This  means  that  we  view  a  portfolio  as 
being  only  as  good  as  its  worst  performance  among  the  scenarios.  The  portfolio  with  the  least  regret 
in  its  worst  performing  scenario  is  our  robust  portfolio. 


Finding  a  Robust  Solution 

This  section  shows  that  by  finding  a  robust  solution,  we  can  avoid  the  pitfalls  of  planning  for  the 
wrong  scenario.  Suppose  we  have  $8  billion,  one  production  line  for  each  weapon,  and  six  years  of 
preparation  time.  We  can  easily  calculate  the  optimal  solution  for  each  of  the  scenarios  shown  in 
Table  12.  These  solutions  are  shown  in  Table  13  below. 

Table  13:  Optimal  Portfolios  for  Each  Scenario,  $8  Billion,  One  Line  and  6  years  of  Preparation 


Scenario 

Optimal  Portfolio 

Munitions  Used 

JDAM 

TLAM 

JSOW 

LGB 

Maverick 

Total 

APC 

240 

2,880 

7,200 

51,000 

10,800 

72,120 

Balanced 

9,240 

2,880 

7,200 

33,000 

10,800 

63,120 

Building 

42,000 

2,880 

6,000 

240 

6,000 

57,120 

Bunker 

240 

2,880 

6,000 

42,000 

6,000 

57,120 

Tank 

240 

2,880 

7,200 

58,200 

6,000 

74,520 

The  total  number  of  munitions  used  represents  the  absolute  best  we  can  do  in  each  scenario,  given 
the  constraints  of  the  budget,  production  and  preparation  time.  The  solutions  are  all  different,  both 
in  terms  of  weapons  mix  and  total  size.  If  we  plan  for  any  one  of  these  wars  we  can  rest  assured  that 
we  can  do  no  better  if  that  war  occurs.  But  the  results  can  be  disastrous  if  we  plan  for  one  scenario 
and  a  different  one  occurs.  Suppose,  for  example,  that  we  plan  for  a  scenario  that  is  very  heavy  in 
mobile  soft  targets  -  an  agile,  mobile  force.  How  does  such  a  solution  fare  if  we  find,  after  six  years, 
that  we  must  fight  a  different  enemy  who  has  entrenched  himself  in  a  complex  of  bunkers  or  has  a 
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larger  army  of  tanks?  I  turns  out  that  the  building  solution  cannot  even  cover  all  targets  in  any  of  the 
other  scenarios.  The  penalty  for  guessing  wrong  is  steep,  as  shown  by  Table  14. 


Table  14:  Performance  of  Optimal  Portfolios  Across  Scenarios  in  Terms  of  Munitions  Used,  $8 
Billion,  One  Line  and  6  Years  of  Preparation 

Denotes  that  portfolio  did  not  cover  all  targets) 


Planned  Portfolio 

APC 

Balanced 

Actual  Scenario 
Building 

Bunker 

Tank 

APC 

72120 

67620 

- 

57120 

- 

Balanced 

- 

63120 

- 

60800 

- 

Building 

- 

- 

57120 

- 

- 

Bunker 

- 

- 

- 

57120 

- 

Tank 

74520 

70020 

74520 

57120 

74520 

Only  by  preparing  for  the  tank  scenario  are  we  assured  of  covering  all  targets  regardless  of  which  war 
actually  occurs.  If  we  had  prepared  for  any  other  war,  we  would  be  unable  to  cover  targets  had 
another  war  occurred!  Even  the  optimal  portfolio  for  the  balanced  scenario  fails  to  cover  targets  in 
three  other  wars.  The  tank  portfolio  is  somewhat  robust  in  the  sense  that  it  covers  targets  in  all  over 
the  wars.  However,  it  uses  30%  more  weapons  than  the  optimal  would  have  used  in  the  building 
scenario.  This  is  the  regret  -  we  could  have  don  30%  better  had  we  known  the  building  scenario 
would  occur.  In  addition,  in  order  to  have  the  tank  portfolio,  we  would  have  needed  to  guess  that 
our  opponent  was  going  to  have  an  army  full  of  tanks.  We  can  do  better  —  by  searching  through  our 
randomly  generated  portfolios  for  a  solution  that  minimizes  the  maximum  regret  across  wars.  This 
search  reveals  the  portfolio  shown  in  Table  15  as  the  most  robust. 


Table  15:  Composition  of  the  Robust  Portfolio,  $8  Billion,  One  Line  and  6  Years  of  Preparation 


JDAM 

TLAM 

JSOW 

LGB 

Maverick 

40003 

1362 

6823 

50227 

9775 

This  portfolio  is  quite  different  from  any  of  the  others  -  it  is  almost,  but  not  quite,  an  additive 
combination  of  the  five  optimals.  It  is  a  compromise  between  the  five  that  allows  for  near  optimal 
performance  in  each.  A  visual  representation  of  the  differences  in  the  five  portfolios  is  shown  below. 
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Figure  10:  Comparing  the  Robust  Portfolio  With  the  Optimal  Portfolios  for  Each  War 
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How  close  to  optimal  performance  do  we  get  with  the  robust  solution?  Table  16  shows  the  percent 
increase  in  munitions  used  to  cover  all  targets  when  compared  to  the  optimal  solution  assuming 
complete  foreknowledge.  For  example,  if  we  choose  the  $8  tank  portfolio  and  the  Building  scenario 
occurs,  we  must  use  30%  more  weapons  to  cover  the  targets  than  we  would  use  if  we  had  planned  for 
the  building  scenario’s  occurrence.  If  we  choose  the  $8  billion  building  portfolio  and  the  tank 
scenario  occurs,  we  cannot  cover  all  the  targets.  If  we  choose  the  robust  portfolio,  we  cover  all  targets 
in  all  wars  and  never  do  more  than  4%  worse  than  the  best  possible  solution  no  matter  which  war 


occurs. 
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Table  16:  Regret  -Percentage  Increase  of  Munitions  Used  Over  the  Best  Case, ,  $8  Billion,  One 

Line  and  6  Years  of  Preparation 


Planned  Portfolio 

APC 

Balanced 

Actual  Scenario 
Building 

Bunker 

Tank 

Worst 

Performance 

APC 

0% 

7% 

- 

0% 

- 

- 

Balanced 

- 

0% 

- 

6% 

- 

- 

Building 

- 

- 

0% 

- 

- 

- 

Bunker 

- 

- 

- 

0% 

- 

- 

Tank 

3% 

11% 

30% 

0% 

0% 

30% 

Robust 

3% 

4% 

3% 

3% 

3% 

4% 

By  choosing  a  robust  solution  we  are  guaranteed  target  coverage  and  a  low  regret  no  matter  which 
war  occurs!  We  do  not  have  to  run  the  risk  of  being  disastrously  unprepared  by  guessing  which  war 
will  occur.  By  searching  through  the  space  of  non-optimal  portfolios,  we  have  found  one  that 
performs  almost  as  well  as  the  optimal  in  every  single  scenario. 

Changing  the  Budget 

As  we  showed  in  the  first  part  of  this  chapter,  changing  the  budget  changes  the  nature  of  the  optimal 
solution.  It  does  not,  however,  change  the  benefits  of  looking  for  a  robust  solution.  Table  17  shows 
the  composition  of  the  optimal  portfolios  for  each  scenario  and  robust  portfolio  when  the  budget  is 
reduced  to  $5  billion.  The  far  right  column  shows  the  number  of  weapons  required  to  cover  all 
targets  by  the  optimals.  As  we  would  expect,  this  number  has  increased  with  the  decrease  in  budget  - 
the  best  possible  solution  is  not  quite  as  good  because  we  can  no  longer  afford  to  buy  many  of  the 
more  expensive  weapons  like  TLAM  and  JSOW. 
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Table  17:  Optimal  and  Robust  Portfolios:  $5  Billion,  One  line,  Six  Years 


JDAM 

TLAM 

JSOW 

LGB 

Maverick 

Munitions  Used 

APC 

6,000 

- 

3,676 

56,286 

10,800 

76,762 

Balanced 

15,000 

283 

7,200 

32,435 

10,800 

65,718 

Building 

42,000 

2,373 

6,000 

1,253 

6,000 

57,626 

Bunker 

6,000 

1,038 

6,000 

39,923 

6,000 

58,961 

Tank 

6,000 

- 

5,487 

60,769 

6,000 

78,256 

Robust 

41,796 

79 

OO 

13,786 

6,000 

- 

Figure  1 1  shows  the  composition  of  each  portfolio  visually.  We  can  see  that  the  robust  portfolio  is 
no  longer  an  additive  compromise  —  at  $5  billion  we  can  not  afford  to  buy  all  the  JDAMs  we  would 
like  in  the  building  optimal  and  all  the  LGBs  we  would  like  for  the  other  optimals.  Instead  a 
compromise  is  reached  where  we  would  buy  enough  JDAMs  to  cover  the  building  scenario,  and  as 
many  LGBs  as  possible  after  that. 
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Figure  11:  Optimal  and  Robust  Portfolios:  $5  Billion,  One  Line,  Six  years 
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Table  18  shows  the  regret  of  each  portfolio  across  the  war  scenarios.  At  $5  billion,  we  face  the  same 
dilemma  as  before  if  we  try  to  guess  which  war  will  occur.  In  most  cases,  planning  for  the  wrong  war 
results  in  us  being  unable  to  cover  all  the  targets.  Again,  the  tank  portfolio  can  cover  targets  in  every 
war,  but  performs  very  poorly  in  the  building  war.  Once  again,  the  robust  portfolio  performs  quite 
well  across  the  range  of  wars,  covering  all  targets  and  removing  the  need  to  guess  which  war  will 
occur.  The  robust  portfolio  has  a  bit  more  regret  in  each  war  at  $5  billion  than  it  did  at  $8  billion  — 
but  it  still  saves  us  from  a  catastrophe  in  any  single  war.  The  lower  budget  makes  it  more  difficult  to 
buy  weapons  that  perform  very  well  against  specific  target  types  and  can  hit  multiple  target  types  - 
like  JSOW.  In  other  words,  the  flexibility  of  the  robust  portfolio  is  limited  because  some  trades 
between  weapons  are  no  longer  affordable.  This  is  exactly  what  happens  with  the  optimal  portfolios 
-  as  the  budget  declines,  optimal  performance  decreases. 
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Table  18:  Regret  -  Percentage  Increase  of  Munitions  Used  Over  the  Best  Case,  $5  Billion,  One  Line, 

and  Six  Years 


Planned  Portfolio 

APC 

Balanced 

Actual  Scenario 
Building 

Bunker 

Tank 

Worst 

Performance 

APC 

0% 

10% 

- 

4% 

- 

10% 

Balanced 

- 

0% 

- 

- 

- 

- 

Building 

- 

- 

0% 

- 

- 

- 

Bunker 

- 

- 

- 

0% 

- 

- 

Tank 

2% 

12% 

36% 

2% 

0% 

36% 

Robust 

8% 

9% 

8% 

6% 

6% 

9% 

Changing  the  Production  Constraint 

We  already  showed  that  increasing  production  capacity  could  improve  the  performance  of  mid  to  low 
budget  portfolios  and  increase  the  maximum  amount  that  can  be  usefully  spend  on  munitions.  In 
this  example,  we  show  that  by  adding  an  extra  line  of  JSOW,  TLAM,  and  Maverick,  we  can  improve 
performance  in  each  scenario  individually  and  by  looking  for  a  robust  portfolio  we  can  improve 
performance  across  all  scenarios.  For  this  example,  we  return  to  a  budget  of  $8  billion.  Table  19 
shows  the  optimal  portfolios  for  each  war  given  an  $8  billion  budget  and  an  extra  production  line  for 
JSOW,  TLAM,  and  maverick.  It  also  shows  the  composition  of  the  most  robust  $8  billion  portfolio 
we  could  find  in  a  search  through  10,000  randomly  generated  portfolios.67  The  far  right  columns 
compare  the  performance  of  the  portfolios  listed  with  the  performance  of  the  optimals  found  in 
Table  13  above  (when  there  is  only  one  line  per  weapon).  As  we  expect,  the  best  possible 
performance  of  an  $8  billion  portfolio  has  improved  with  a  slackening  of  the  production  constraint. 
When  commanders  are  allowed  to  buy  more  TLAMs,  JSOWs  and  Mavericks,  performance  improves. 
Figure  12  shows  the  2  line  portfolios  visually. 


67  A  new  set  of  portfolios  was  generated  here,  based  on  6  years  worth  of  production  from  one  line  of  LGB’s  and  JDAMs  and  two 
lines  of  JSOWs,  TLAMs,  and  Mavericks. 
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Table  19:  Optimal  and  Robust  Portfolios:  $8  Billion,  Two  Lines  for  JSOW,  TLAM,  and  Maverick, 

Six  Years  Preparation 


JDAM 

TLAM 

JSOW 

LGB 

Maverick 

Munitions  Used 

Munitions  Used  (From 
Table  12) 

1  Line 

APC 

6,000 

1,253 

14,400 

21,494 

21,600 

64,747 

72,120 

Balanced 

15,000 

2,586 

14,400 

10,729 

15,000 

57,715 

63,120 

Building 

36,480 

5,760 

6,000 

- 

6,000 

54,240 

57,120 

Bunker 

6,000 

4,875 

6,000 

32,251 

6,000 

55,126 

57,120 

Tank 

6,000 

2,281 

14,400 

42,839 

6,000 

71,520 

74,520 

Robust 

44,410 

696 

10,751 

39,110 

16,377 

Figure  12:  Optimal  and  Robust  Portfolios:  $8  Billion,  Two  Lines  for  JSOW,  TLAM,  and  Maverick, 

Six  Years  Preparation 
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Although  the  performance  of  the  optimal  solutions  has  improved  with  the  added  lines,  we  still  face 
the  same  danger  if  we  plan  only  for  one  scenario.  Table  20  shows  the  regret  of  preparing  for  each 
scenario  in  terms  of  the  number  of  weapons  used.  Once  again,  we  find  that  preparing  for  any  specific 
scenario  will  leave  us  unable  to  cover  targets  should  a  different  scenario  occur.  Again,  the  optimal 
tank  portfolio  can  cover  all  targets  in  each  scenario  but  does  poorly  in  the  building  scenario.  But  the 
robust  portfolio  covers  all  targets  in  all  scenarios  and  does  no  more  than  9%  worse  than  the  optimal 
in  any  single  scenario. 
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Table  20:  Regret  -  Percentage  Increase  of  Munitions  Used  Over  the  Best  Case 

Denotes  that  portfolio  could  not  cover  all  targets) 


Planned 

Portfolio 

APC 

Balanced 

Actual  Scenario 

Building 

Bunker 

Tank 

Worst  Performance 

APC 

0% 

8% 

- 

- 

- 

- 

Balanced 

- 

0% 

- 

- 

- 

- 

Building 

- 

- 

0% 

- 

- 

- 

Bunker 

1% 

3% 

- 

0% 

- 

- 

Tank 

10% 

12% 

28% 

5% 

0% 

28% 

Robust 

8% 

6% 

9% 

9% 

6% 

9% 

Thus  far  we  have  compared  the  2  line  robust  solution  with  the  2  line  optimal  solutions.  It  has  more 
regret  than  did  the  1  line  robust  solution  compared  to  the  one  line  optimals.  How  does  the  two  line 
robust  solution  compare  with  the  one  line  robust  solution?  Do  the  added  lines  improve  the 
performance  of  the  robust  solution?  We  would  expect  so.  Table  21shows  us  that  this  is  to  some 
extent  true. 


Table  21:  Difference  in  Performance:  Robust  with  One  Line  and  Robust  With  Added  Lines 


APC 

Balanced 

Building 

Bunker 

Tank 

Robust  -  One  Line 

74,390 

65,339 

58,638 

58,638 

76,906 

Robust  -  Added  Lines 

69,740 

61,429 

59,304 

60,053 

75,593 

Difference 

-6% 

-6% 

1% 

2% 

-2% 

The  robust  portfolio  with  added  lines  does  slightly  better  in  the  APC  and  balanced  scenarios, 
marginally  better  in  the  Tank  scenario,  and  marginally  worse  in  the  building  and  bunker  scenarios. 
The  reason  for  the  worse  performance  in  the  Building  and  Bunker  scenarios  is  likely  as  result  of  either 
sampling  error  or  the  fact  that  the  improvements  gained  from  adding  munitions  used  in  the  other 
scenarios  outweighed  the  slight  decrease  in  performance. 

Changing  the  Preparation  Time 

Suppose  we  are  concerned  about  being  surprised.  What  if  the  war  occurs  before  we  expect?  As  we 
showed  in  the  first  part  of  this  illustrative  analysis,  a  surprise  war  can  drastically  reduce  the 
effectiveness  of  a  portfolio.  This  section  will  show  that  by  searching  for  robust  portfolios  we  can  find 
a  solution  that  works  well  across  wars,  even  in  the  event  of  an  early  war.  In  this  example,  we  have  a 
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budget  of  $8  billion,  one  production  line  for  each  weapon,  and  two  year’s  less  time  to  prepare.  Table 
22  shows  the  optimal  and  robust  portfolios  for  each  war  scenario,  given  4  years  preparation  time,  an 
$8  billion  budget,  and  one  production  line  for  each  weapon.  The  far  right  column  shows  how  the 
optimal  solutions  given  6  years  of  preparation  time  fare  when  the  war  actually  occurs  in  4  years. 

None  of  the  6  year  optimals  can  cover  all  targets  in  4  years.  Figure  13  visually  compares  the  optimal 
portfolios  with  the  robust  portfolios. 


Table  22:  Composition  Of  Robust  and  Optimal  Solutions:  $8  B,  1  Line,  4  Years  of 

Preparation 


JDAM 

TLAM 

JSOW 

LGB 

Maverick 

Munitions 

Used 

Munitions  Used  -  6 
Year  Optimals 

APC 

2,160 

1,920 

4,800 

60,000 

7,200 

76,080 

- 

Balanced 

11,160 

1,920 

4,800 

42,000 

7,200 

67,080 

- 

Building 

42,000 

1,920 

4,800 

3,960 

6,000 

58,680 

- 

Bunker 

2,160 

1,920 

4,800 

43,800 

6,000 

58,680 

- 

Tank 

2,160 

1,920 

4,800 

61,800 

6,000 

76,680 

- 

Robust 

54,994 

1,714 

4,800 

59,420 

7,200 

Figure  13:  Optimal  and  Robust  Portfolios:  $8  B,  Two  Lines  for  JSOW,  TLAM,  and 

Maverick,  Six  Years  Preparation 
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Table  23  shows  what  we  saw  for  both  the  budget  and  production  examples  above.  The  robust 
portfolio  insulates  us  from  the  risk  of  planning  for  the  wrong  war  by  containing  weapons  that 
perform  well  across  the  range  of  possible  wars.  In  this  case,  there  is  almost  no  difference  in 
performance  between  the  robust  portfolio  and  the  optimal  portfolio  for  each  war.  But,  if  we  had 
chosen  to  prepare  for  only  one  of  the  wars,  we  would  have  run  the  risk  of  being  unable  to  cover 
targets  should  a  different  war  occur. 


Table  23:  Regret  -  Percentage  Increase  of  Munitions  Used  Over  the  Best  Case  -  $8  B,  One  Line,  4 

Years  of  Preparation  Time 

Denotes  that  portfolio  could  not  cover  all  targets) 


Planned  Portfolio 

APC 

Balanced 

Actual  Scenario 
Building 

Bunker 

Tank 

Worst 

Performance 

APC 

0.0% 

6.7% 

- 

0.0% 

- 

- 

Balanced 

- 

0.0% 

- 

0.0% 

- 

- 

Building 

- 

- 

0.0% 

- 

- 

- 

Bunker 

- 

- 

- 

0.0% 

- 

- 

Tank 

0.8% 

7.6% 

30.7% 

0.0% 

0.0% 

30.7% 

Robust 

0.3% 

0.3% 

0.4% 

0.4% 

0.3% 

0.4% 

A  Note  About  the  Tank  Scenario  Solutions 

The  tank  scenario  optimal  portfolios  tend  to  perform  fairly  well  across  scenarios  independent  of 
changed  in  the  constraints.  The  reason  for  this  is  likely  the  munitions  intensive  nature  of  the  tank 
scenario.  What  I  mean  is  that  the  tank  scenario  always  requires  more  weapons,  therefore  the  optimal 
solution  to  the  scenario  will  have  more  weapons  to  work  with  should  another  scenario  arise.  The  fact 
remains  that  without  having  done  this  analysis  we  never  would  have  known  that  the  tank  solutions 
tend  to  be  somewhat  robust. 


CHAPTER  EIGHT 

Conclusions  and  Limitations 


Benefits  of  This  Approach 

Recap 

We  opened  this  paper  by  expressing  the  DoD’s  desire  to  implement  capabilities-based  planning, 
describing  the  current  PGM  requirements  process,  and  presenting  the  official  critiques  of  that  system. 
We  argued  that  a  capabilities-based  approach  would  need  to  take  into  account  the  uncertainties  that 
are  inherent  to  the  problem  within  the  context  of  economic  constraints.  The  specific  constraints  and 
uncertainties  examined  in  the  illustrative  analysis  provided  here  are  the  budget,  the  number  of 
production  lines  available  for  each  munition,  the  amount  of  time  we  have  to  prepare  for  a  war,  and 
the  uncertainty  of  the  war  itself.  We  showed  that  due  to  the  nature  of  the  uncertainties,  traditional 
methodologies  do  not  solve  this  problem.  We  cannot  place  a  probability  on  which  war  will  occur  or 
how  long  we  will  have  to  prepare  for  it.  We  therefore  turned  to  the  methodologies  of  exploratory 
analysis  and  robust  decision-making  and  showed  the  usefulness  of  the  two  methodologies  in 
answering  the  following  policy  questions: 

1.  How  do  the  constraints  of  budget  and  available  production  lines  and  the  uncertainty  of 
preparation  time  before  a  war  affect  the  optimal  munitions  portfolio  when  we  assume  we 
know  which  war  scenario  will  occur? 

2.  Given  that  we  do  not  know  which  war  will  occur,  how  do  we  find  a  portfolio  of 
munitions  that  works  well  across  a  variety  of  scenarios,  taking  into  account  the 
aforementioned  uncertainty  and  constraints? 

These  are  questions  that  high-level  policymakers  ask  the  requirements  office  right  now.  A  general  in 
the  Joint  staff  would  want  to  know  how  a  changing  the  budget  affects  his  ability  prepared  for  a  war 
(or  wars).  The  Secretary  of  Defense  would  want  to  know  that  we  are  prepared  to  fight  a  multitude  of 
different  kinds  of  wars.  The  current  requirements  and  budgeting  processes  cannot  answer  them  as 
well  as  many  would  like. 

Exploratory  Modeling 

In  the  first  half  of  the  last  chapter  we  used  exploratory  analysis  to  show  the  effects  of  changing  the 
constraints  of  budget  and  available  production  lines  as  well  as  the  uncertainty  of  preparation  time  for 
a  single  war.  We  found  that  planning  for  only  one  assumed  future  can  lead  to  poor  performance 
when  something  else  occurs.  A  combatant  commander  prefers  different  portfolios  of  weapons  at 
different  budget  levels.  When  we  make  plans  assuming  that  we  have  no  budget  limit,  we  choose  a 
portfolio  that  performs  very  poorly  if  it  is  later  constrained  by  a  budget.  In  many  cases,  the  binding 
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constraint  on  a  commander’s  preferred  portfolio  is  not  the  budget,  but  rather  the  number  of  certain 
types  of  munitions  that  we  can  produce.  We  found  that  increases  in  production  capacity  can 
improve  performance  at  many  budget  levels,  and  make  expensive  portfolios  more  viable.  If  a  war 
occurs  with  less  warning  than  we  expect,  we  may  be  stuck  without  enough  munitions  to  cover  all 
targets.  We  found  that  by  examining  the  effects  of  changing  the  preparation  time,  we  can  find 
specific  portfolios  that  guard  against  an  early  war. 

By  using  the  technique  of  exploratory  analysis  we  are  able  to  show  the  tradeoffs  that  are  made  as  each 
variable  changes.  This  information  is  useful  to  policymakers  because  it  allows  them  to  understand 
exactly  how  their  decisions  (and  the  decisions  of  others)  affect  the  outcome  under  a  wide  variety  of 
circumstances.  In  the  DPRK  analysis  above,  we  showed  that  changes  in  the  budget  and  number  of 
production  lines  can  dramatically  improve  performance.  The  degree  of  improvement  depends  on  the 
current  level  of  spending  and  production.  Using  an  analysis  similar  to  the  one  we  have  provided 
here,  DoD  officials  can  make  a  solid  case  to  congress  that  more  money  or  production  is  needed  and 
explain  why.  They  can  ask  themselves  how  they  might  change  their  own  portfolio  to  prepare  for  the 
event  of  a  surprise  war,  and  understand  why  specific  changes  help.  In  addition,  these  kinds  of 
analyses  may  be  done  relatively  quickly. 

Robust  Decision-Making 

In  the  second  half  of  the  last  chapter  we  used  robust  decision-making  techniques  to  find  portfolios 
that  work  well  across  a  wide  variety  of  disparate  wars.  The  fact  we  cannot  predict  which  war  might 
occur  makes  it  impossible  to  determine  an  optimal  portfolio  of  weapons,  even  when  other  constraints 
and  uncertainties  are  fixed.  Planning  for  only  a  single  war  can  be  catastrophic  should  a  war  with  very 
different  attributes  occur.  We  showed  that  by  searching  for  a  robust  solution,  it  is  possible  to  find  a 
portfolio  that  does  not  perform  optimally  in  any  single  war,  but  performs  almost  as  well  as  the 
optimal  portfolio  across  all  wars.  We  also  showed  that  robust  decision-making  techniques  can  also  be 
used  to  examine  how  the  best  robust  portfolio  changes  with  changes  to  the  constraints  of  budget  and 
production  capacity  and  the  uncertainty  of  preparation  time.  We  found  that  robust  portfolios  do 
exist  and  that  these  portfolios  do  change  as  the  constraints  and  other  uncertainties  are  altered. 

Such  information  is  useful  to  policymakers  because  it  demonstrates  the  nature  of  the  uncertainty  they 
face  and  provides  a  way  to  plan  even  in  the  face  of  it.  That  is,  we  are  shown  the  consequences  of  an 
incorrect  prediction  —  planning  for  one  war  and  having  another  one  occur.  But  we  are  also  shown 
that  it  is  possible  to  create  a  portfolio  that  is  flexible  enough  to  perform  well  across  all  wars.  In 
addition,  we  can  vary  the  constraints  and  uncertainties  in  order  to  see  how  they  shape  the  robust 
solution.  As  before,  we  can  see  the  degree  to  which  changes  in  the  constraints  of  budget  and 
production  and  the  uncertainty  of  preparation  time  affect  our  ability  to  choose  a  good  portfolio. 
Except,  in  this  case,  a  good  portfolio  is  robust  across  scenarios  rather  than  optimal  for  one. 
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Implementation  and  Institutional  Issues 

There  are  two  ways  this  kind  of  approach  could  be  implemented:  either  as  a  replacement  to  the 
current  system  or  as  a  secondary  process  that  operates  outside,  but  in  conjunction  with  the  budgeting 
and  requirements  processes.  The  first  option  is  not  immediately  practical.  There  are  a  plethora  of 
offices  inside  the  DoD  and  armed  services  whose  sole  purpose  is  to  calculate  the  annual  requirement, 
and  maintain  the  manpower  and  models  needed  to  do  so.  Furthermore,  there  are  a  large  number  of 
outside  contractors  whose  main  source  of  income  is  derived  from  supporting  the  process.  To 
completely  dismantle  and  restructure  a  planning  system  of  this  size  is  a  policy  problem  in  and  of  itself 
that  would  require  considerable  organizational  and  political  expertise  to  solve.  It  is  well  beyond  the 
scope  of  this  dissertation.  It  would  involve  taking  apart  and  changing  an  organizational  structure  that 
is  embedded  deeply  within  each  layer  of  the  defense  department,  and  would  greatly  affect  how  the 
defense  contracting  industry  does  business.  In  addition,  it  would  be  a  mistake  to  completely  replace 
the  current  system  with  one  that  has  not  been  thoroughly  tested  in  the  actual 
planning/budgeting/ programming  environment. 

An  appealing  alternative  would  be  to  house  such  an  approach  within  an  independent  office  inside  the 
office  of  the  Joint  Staff  of  the  DoD  or  inside  the  services  themselves.  The  staff  would  immediately 
benefit  by  having  a  way  to  link  the  requirements  and  budgeting  processes.  An  advantage  of  the 
approach  outlined  here  is  the  speed  with  which  the  models  can  be  run  and  rerun  with  new  data. 

Such  an  office  could  use  the  estimate  generated  by  the  current  requirements  process  as  an  initial  input 
to  models  like  the  ones  shown  here,  which  could  then  assess  how  the  requirement  might  perform 
under  varying  circumstances  and  economic  constraints.  This  would  provide  the  staff  with  a  way  of 
quickly  understanding  the  strengths  and  weaknesses  of  the  initial  requirement  and  making 
adjustments  based  on  budgetary  and  other  concerns  within  the  narrow  time  frame  allowed  by  the 
budgeting  process.  Furthermore,  this  would  be  a  good  way  to  see  how  a  capabilities-based  approach 
performs  in  practice  relative  to  the  current  processes.  We  would  not  need  to  make  premature 
changes  to  a  system  that,  while  flawed,  has  served  us  well  for  decades.  If  the  results  are  good,  further 
modifications  and  study  would  likely  be  needed  prior  to  a  larger  scale  application. 


Improvements  and  Extensions 

Our  demonstration  of  this  approach  has  shown  that  applying  the  methodologies  of  robust  decision¬ 
making  to  the  problem  of  munitions  planning  to  the  problem  of  PGM  requirements  generation,  it  is 
possible  to  not  only  do  better,  but  also  to  show  why  we  might  need  more  or  less  money/production 
and  understand  why  we  would  choose  some  mixes  over  others.  However,  there  are  many  ways  the 
illustrative  analysis  presented  here  could  be  improved  upon.  We  can  examine  robustness  across 
multiple  dimensions  and  mission  goals.  We  can  include  current  the  current  stockpile  in  our 
calculations.  We  can  include  more  sophisticated  production  strategies  that  affect  the  flow  of 
munitions  as  well  as  the  stock  in  our  analysis.  We  can  include  a  more  sophisticated  war  model. 
Finally,  we  must  use  more  accurate  data  on  weapon  effectiveness  before  we  can  begin  to  make  policy 
based  on  our  results. 
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Robustness  Across  Multiple  Dimensions 

In  this  illustrative  analysis,  we  only  considered  robustness  across  a  single  dimension  —  the  war 
scenario.  We  looked  for  portfolios  that  performed  well  across  all  war  scenarios  at  various  fixed  levels 
of  budget,  production  and  warning  time.  It  is  possible  to  search  for  a  robust  solution  across  multiple 
dimensions.  For  example,  we  could  search  for  portfolios  are  robust  to  both  preparation  time  and  the 
war  scenario.  This  search  would  entail  testing  portfolios  across  all  possible  combinations  of  the  war 
scenarios  and  warning  times.  It  would,  however,  be  useful  to  policymakers  who  are  worried  about 
the  potential  for  a  surprise  war,  but  also  want  to  be  prepared  very  well  for  a  war  that  occurs  a  long 
way  off. 

Investigating  Robustness  Across  Mission  Goals 

We  assumed  that  the  goal  of  a  war  is  to  cover  all  the  targets  and  use  a  small  number  of  munitions  in 
doing  so.  In  this  day  and  age,  we  can  expect  to  fight  wars  with  widely  varying  mission  goals.  For 
example,  we  might  fight  a  war  as  a  tank  vs.  tank  melee,  a  counter-insurgency,  a  ‘shock  and  awe’ 
campaign,  or  a  series  of  urban  battles  depending  on  our  goals  and  the  tactics  of  our  enemy.  Each 
type  of  war  brings  with  it  a  different  set  of  objectives  and  performance  measures.  In  choosing  to 
examine  the  uncertainty  of  target  mix,  we  neglected  to  consider  the  uncertainty  of  mission  goal. 
However,  we  believe  it  would  not  be  difficult  to  incorporate  into  an  additional  analysis. 

Inclusion  of  Current  Stockpile 

The  model  used  here  did  not  take  into  account  the  currently  owned  stockpile  -  rather  it  assumed  that 
we  were  building  a  stockpile  from  scratch.  The  reality  is  that  yearly  investments  in  munitions  act  as 
marginal  changes  to  the  stockpile  we  already  have.  Inclusion  of  the  current  inventory  would 
constrain  the  optimal  solutions,  and  make  some  strategies  more  desirable  than  others  in  ways  that 
cannot  necessarily  be  predicted  in  advance.  However,  one  of  the  benefits  of  using  a  linear 
programming  technique  is  the  ease  with  which  we  may  add  to  the  model  a  base  inventory  that  has 
already  been  paid  for. 

Investigation  of  Strategies  that  Affect  the  Flow  of  Munitions  as  well  as  the  Stock 

We  considered  only  one  production  policy  —  the  number  of  lines  available  -  and  we  considered  it  as  a 
constraint  rather  than  a  policy  lever.  In  Chapter  Two,  we  pointed  out  that  system  performance 
could  be  improved  by  making  investments  in  production  technologies  and  options  in  addition  to 
physical  stock.  However,  we  did  not  have  time  to  thoroughly  investigate  such  policies.  Production 
line  sharing,  surge  production,  and  options  all  have  the  potential  to  improve  performance  across 
scenarios  by  enabling  us  to  create  more  flexible  portfolios.  An  analysis  examining  the  effects  of  these 
policies  could  be  revealing  to  the  PGM  community. 

More  Sophisticated  War  Model 

Our  war  model  was  very  simple.  It  matched  five  munitions  to  four  target  types.  In  reality,  there  are 
many  different  types  of  targets  and  munitions.  There  is  also  more  to  a  war  than  simply  hitting  all  of 
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the  targets.  The  timing  of  attacks  and  order  in  which  targets  are  hit  can  make  a  crucial  difference  in 
the  outcome.  We  dealt  only  with  the  uncertainties  of  target  type  and  war  timing.  There  are  many 
more  uncertainties  that  can  affect  a  munitions  portfolio’s  performance.  For  example,  the  weather 
and  terrain  type  can  drastically  affect  the  effectiveness  of  munitions.  The  use  of  a  more  sophisticated 
war  model  was  considered,  but  deemed  unnecessary  for  the  purposes  of  this  demonstration. 

Better  Munitions  Effectiveness  Data 

The  true  effectiveness  (pK)  of  each  munition  against  each  target  type  is  classified.  For  this  reason,  I 
had  to  make  an  educated  guess  about  how  effective  each  weapon  is  against  each  target.  In  doing  so,  I 
reviewed  unclassified  data  on  usage  in  past  wars,  and  documents  describing  each  munition’s 
attributes.  In  all  likelihood,  the  relative  effectiveness  of  the  weapons  against  each  target  is  accurate. 
But  there  may  be  errors  and  the  overall  precision  of  these  weapons  may  be  over  or  underestimated  in 
the  illustrative  analysis  presented  above.  In  addition,  this  approach  did  not  consider  traditional 
munitions  as  an  alternative  to  PGMs  (a  change  that  could  be  made  easily).  As  a  result,  the  models 
provided  here  should  not  be  used  to  determine  specific  requirements  or  policies.  It  would,  however, 
be  a  simple  matter  to  replace  the  effectiveness  data  with  the  actual  numbers.  The  models  could  be 
rerun  in  a  day’s  time.  Changing  the  effectiveness  data  would  not  change  the  major  conclusions  of 
this  report.  It  would,  however,  change  the  composition  of  optimal  and  robust  portfolios  and  the 
specific  tradeoffs  preferred  by  commanders  under  various  circumstances. 
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Notional  Number  of  Weapons  Needed  to  Cover  a  Single  Target 

(Used  as  model  inputs  —  ‘covering’  a  target  is  having  a  95%  chance  of  destroying  it) 


Mobile  Hard 

Mobile  Soft 

Fixed  Hard 

Fixed  Soft 

JDAM 

4 

4 

8 

2 

TLAM 

- 

- 

1 

1 

JSOW 

2 

2 

3 

2 

LGB 

3 

3 

2 

3 

Maverick 

3 

2 

- 

- 

Weapon  Cost  and  Production  Data 

JDAM 

TLAM 

JSOW  LGB 

Maverick 

Weapon  Unit  Cost  ($M) 

0.02 

.88 

.21  .05 

.13 

Monthly  Production  of  a  Single  Assembly 

3000 

40 

100  1600 

150 

Line 

Source:  Estimates  derived  from  Selected  Acquisition  Reports  for  each  munition. 

Ranking  of  Weapi 

ons  Based  on  Number  Needed  to  Cover  a  Target 

Best 

2nd  Best 

3rd  Best 

4th  Best 

Mobile  Hard 

JSOW 

LGB /Maverick 

JDAM 

Mobile  Soft 

J  SOW/Maverick 

LGB 

JDAM 

Fixed  Hard 

TLAM 

LGB 

JSOW 

JDAM 

Fixed  Soft 

TLAM 

JDAM/JSOW 

LGB 

Ranking  of  Weapons  Based 

on  Cost  per  Covered  Target 

Best 

2nd  Best 

3rd  Best 

4th  Best 

5th  Best 

Mobile  Hard 

JDAM 

LGB 

Maverick 

JSOW 

Mobile  Soft 

JDAM 

LGB 

Maverick 

JSOW 

Fixed  Hard 

LGB 

JDAM 

JSOW 

TLAM 

Fixed  Soft 

JDAM 

LGB 

JSOW 

HARM 

TLAM 

Hard  SO 

JSOW 

TLAM 

Soft  SO 

JSOW 

TLAM 
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Here  are  the  actual  “Cost  per  target  covered”  numbers  used  to  create  the  above  ranking: 


Unit  Cost  Per  Target  Covered  ($M) 


JDAM 

TLAM 

JSOW 

LGB 

Maverick 

Mobile  Hard 

0.08 

- 

0.42 

0.15 

0.38 

Mobile  Soft 

0.08 

- 

0.42 

0.15 

0.25 

Fixed  Hard 

0.16 

0.88 

0.63 

0.10 

- 

Fixed  Soft 

0.04 

0.88 

0.42 

0.15 

- 

Weapon  Unit  Cost  ($M) 

0.02 

0.88 

0.21 

0.05 

0.13 

APPENDIX  B 

Munitions  Considered  In  This  Report _ 

AGM-65  Maverick 

The  AGM-65,  commonly  known  as  Maverick,  first  entered  service  in  the  mid  60s.  The  weapon  has 
been  continuously  upgraded  since  then.  It  is  a  rocket-powered  missile,  originally  designed  for  use 
against  tanks  and  hardened  targets.  Variants  of  the  maverick  use  three  different  targeting  systems. 

The  A,  B,  H,  J  and  K  variants  use  a  TV  tracker  that  locks  onto  a  television  picture  that  the  pilot  takes 
in  the  cockpit.  The  missile  then  homes  in  on  that  target.  The  F,  F  and  G  variants  use  an  imaging 
infrared  (HR)  seeker.  This  seeker  operates  similarly  to  the  TV  guidance,  except  that  it  locks  onto  an 
infrared  image  rather  than  a  visual  one.  These  variants  are  designed  to  operate  better  in  low  visibility 
conditions.  The  E  variant  is  guided  by  a  semi-active  laser.  This  missile  requires  a  separate  laser, 
either  on  the  aircraft  or  operated  by  a  third  party,  to  mark  the  target.  The  missile  then  homes  in  one 
the  laser  energy  reflected  from  the  target.  According  to  Jane’s  Online,  over  30,000  AGM-65  A/Bs 
have  been  produced  and  over  35,000  of  the  other  variants  have  been  ordered.  In  addition,  many  of 
the  older  models  have  been  and  are  continuing  to  be  upgraded. 

Joint  Direct  Attack  Munition  (JDAM) 

The  JDAM  family  of  munitions  is  the  result  of  a  joint  Navy/Air  Force  program  to  find  inexpensive 
ways  to  deliver  conventional  bombs  with  increased  accuracy.  The  program  was  begun  in  the  late 
1980s.  JDAM  is  a  kit  that  is  added  to  a  conventional  bomb.  The  kit  is  comprised  of  a  GPS  guidance 
system  and  a  set  of  steering  fins.  While  in  flight,  the  fins  move  to  home  the  bomb  into  a 
programmed  set  of  GPS  coordinates.  JDAM  comes  in  4  variants.  The  major  difference  between 
variants  is  the  size  of  the  bomb  the  kit  is  added  to.  The  GBU-31  fits  on  a  2000  lb  general  purpose  or 
penetrating  warhead,  the  GBU-32  Fits  on  a  1000  lb  general  purpose  warhead,  the  GBU-35  fits  on  a 
1000  lb  general  purpose  warhead,  and  the  GBU-38  fits  on  a  500  lb  general  purpose  warhead.  As  of 
June  2004,  over  100,000  JDAMs  had  been  produced.  One  advantage  of  the  JDAM  kit  is  its  price  - 
it  costs  about  $30,000  per  kit  —  very  cheap  compared  to  other  precision-guided  weapons.  In 
addition,  it  can  be  used  against  any  kind  of  target.  However,  it  is  not  as  accurate  as  other  PGMs. 
Jane’s  online  reports  a  42.9  ft  circular  probable  error,  meaning  that  the  Air  Force  officially  classifies 
the  weapon  as  “near  precision  weapon.”  The  JDAM  kit  does  not  provide  any  power  to  the  warhead 
-  it  only  assists  the  glide. 

AGM-154  Joint  Standoff  Weapon 

JSOW  is  the  result  of  a  joint  program  by  the  USAF  and  Navy  to  develop  an  inexpensive  precision 
standoff  weapon.  The  program  was  begun  in  1992,  and  low  rate  production  began  in  1998.  The 
JSOW  is  an  unpowered  glide  bomb,  fitted  with  wings  like  an  aircraft,  which  has  the  ability  to  carry 
and  dispense  a  variety  of  sub  munitions.  The  JSOW  is  intended  to  be  a  partial  replacement  for  a 
variety  of  older  PGMs.  There  are  currently  three  major  variants  of  the  JSOW,  each  capable  of 
destroying  a  different  target  type.  The  AGM-154A  carries  145  combined  effect  bomblets  for 
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destroying  soft  targets,  the  AGM-154B  carries  six  bomb  dispensers  (each  of  which  can  dispense  four 
anti  tank  bomblets),  and  the  AGM-154C  carries  a  single  warhead  designed  specifically  for 
penetrating  fixed  hardened  targets.  All  JSOW  variants  carry  a  GPS  for  guidance,  and  the  C  variant 
also  has  the  ability  to  be  guided  by  an  imaging  infrared  seeker.  Because  it  is  a  glide  bomb,  the  range 
of  the  JSOW  varies  depending  on  the  altitude  at  which  it  is  dropped.  A  variety  of  sources  cite  ranges 
of  anywhere  between  5  miles  to  40  miles  (from  a  high  altitude  launch).  In  1994,  Raytheon 
developed  a  powered  version  of  the  JSOW,  with  a  reported  range  of  120  miles  —  but  the  weapon  was 
never  ordered.68  However,  the  capability  to  produce  the  powered  version  remains.  The  JSOW 
carries  a  set  of  preprogrammed  targets,  but  may  also  be  retargeted  manually  from  the  cockpit.  About 
19,000  JSOWs  have  been  order  by  the  Navy,  Air  Force,  and  Marine  Corp,  although  it  was  reported 
in  January  2003  that  the  Air  Force  had  decided  to  abandon  the  project  in  favor  of  the  Wind 
Corrected  Munitions  Dispenser.  However,  that  system  has  been  marked  for  cancellation  in  the  2006 
budget. 

UGM-109  Tomahawk  Land  Attack  Missile  (TLAM) 

The  Tomahawk  is  a  truly  long-range  standoff  weapon,  fired  from  a  ship  or  submarine.  It  comes  in 
four  variants  —  the  TLAM-N,  designed  to  carry  nuclear  warheads;  the  TASM,  designed  to  destroy 
ships;  the  TLAM-C,  designed  to  destroy  conventional  fixed  targets,  and  the  TLAM-D,  which  carries 
conventional  sub  munitions.  Range  varies  by  type,  but  the  conventional  variants  have  a  reported 
range  of  500  nautical  miles.  The  missile  launches  from  its  tube  like  a  rocket.  After  launch,  wings 
extend  from  the  sides  of  the  weapon,  and  it  then  cruises  under  the  power  of  a  turbofan  jet  engine. 
The  missile  then  follows  a  preplanned  route  to  its  target  using  on  board  navigational  systems  and 
terrain  mapping  guidance.  As  it  nears  the  target,  the  missile  switches  to  a  terminal  guidance  system 
that  matches  a  digital  image  of  the  target  with  images  from  an  onboard  camera.  Recent  upgrades 
allow  the  missile  to  be  retargeted  during  flight  from  a  satellite  uplink.  The  first  Tomahawk  entered 
service  in  1983,  with  a  planned  buy  of  approximately  2,500  conventional  TLAMs.  The  actual 
number  held  is  stock  is  closely  guarded.  There  were  many  unconfirmed  reports  of  a  serious  shortage 
after  Operation  Iraqi  Freedom. 

Paveway  II  and  III,  Laser  Guided  Bombs  (LGB) 

The  Paveway  system  is  a  bomb  kit,  much  like  JDAM,  except  that  it  is  guided  to  its  target  by  a  laser. 
There  are  two  pieces  to  the  kit.  The  first  piece  bolts  on  to  the  front  of  the  bomb  and  provides 
guidance  and  control.  This  piece  carries  the  laser  seeker  and  four  fins  used  for  control.  The  second 
piece  bolts  on  to  the  back  of  the  bomb  and  contains  four  fixed  wings  that  stabilize  the  bomb  during 
freefall.  The  original  Paveway  entered  service  in  the  late  1960s.  Paveway  II  entered  service  in  1 976  — 
with  major  improvements  in  electronics,  guidance,  and  maneuverability.  Paveway  III  entered  service 
in  1986  —  with  the  major  upgrades  of  low  altitude  and  longer  standoff  range  capability.  On  all 
Paveway  systems,  the  laser  guidance  requires  a  target  to  be  marked  by  a  laser  designator  that  may  be 
located  either  on  the  aircraft  or  a  third  party  on  the  ground  or  in  the  air.  Recent  upgrades  have 
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added  a  GPS  guidance  system  that  allows  for  better  performance  in  low  visibility  conditions.  The 
bomb  is  guided  near  the  target  using  the  GPS,  and  then  switches  to  a  laser  seeker  for  terminal 
guidance.  There  are  at  least  7  different  variants  of  the  Paveway.  The  major  difference  between 
variants  is  the  warhead  size.  Two  exceptions  are  the  GBU-27  and  GBU-28  that  were  designed 
specifically  for  deep  bunker  penetration.  Janes  Online  reports  that  the  Air  Force  used  8,400  LGBs 
during  the  Gulf  war,  and  still  had  an  inventory  of  approximately  27,000  afterwards. 


